February 2007

Monthly Archive

Curtail complexity with a rules engine

Complexity can manifest itself within a software application in a number of hip ways, including dependency management (i.e. 3rd party libraries required for runtime, etc), architectural adherence patterns (think old style EJBs), and even coding constructs (in particular, excessive use of conditionals). When it comes to coding constructs, the resulting complexity is often related to the problem being solved. For example, imagine a recommendation wizard for sales associates selling hip disco LPs. Quite simple, right? You have two groovy choices– anything from Donna Summer or the Bee Gees. If only life were this easy, eh?

Now imagine a recommendation wizard for smokin’ sales associates trying to move beer. Now that’s more real, isn’t it? Imagine the store is trying to move (i.e. sell to customers) seven different types of beers– all varying in taste and characteristics. The store wants to develop an application that walks someone through a series of questions and based upon their answers, will recommend one of seven beers. Think of this application as an expert beer system– and while it may start with only seven beers, over time more beers will be added, especially if the system proves itself to move beers efficiently. What’s more, you the developer aren’t a beer aficionado (i.e. a domain expert on beers)– your job is to make an application that beer experts can modify so customers can pick beers more easily.

Logically, you can build a hip beer expert system with a couple of conditionals– if you like this characteristic, then you should buy this beer, right? In pseudocode, your logic could look like this (after you’ve had a beer session with the beer experts):

Do you like a light beer or a dark beer?
 if light beer:
  Do you like crisp, smooth beers or more prefer a more hoppy one?
   if crisp:
    then Pils
   else:
    Do you like light hops or more aggressive hops?
      if light:
       then Pale Ale
      else:
       IPA
else:
  Do you like the taste of coffee?
   if yes:
     Chocolate Stout
   else:
     Do you like spiciness?
       if yes:
         try Winter Ale
       else:
          Do you like high alcohol content?
            if yes:
              try an Eisbock
            else:
              try a Lager

This particular block of code (which enables one to pick one of seven beers and is by no means an accurate expert system), if isolated in a method, would have a cyclomatic complexity of at least 13, which presents a challenge– methods over 10, with conditional nesting, are havens for defects, especially if this code changes often. What if next week, the Pilsner brand is sold out? You’ll need to modify the logic to select perhaps another type of beer. In fact, the logic may not be as easy as replacing the Pilsner with another neat-o beer– it may involve a new series of questions.

It turns out that in these scenarios, a rules engine may actually be beneficial– in fact, rules engines (or expert systems) are well suited to replace excessive if, else, switch logic, especially if that logic is the domain of non-technical experts (in the case above, the beer experts haven’t a clue about coding nor hygiene, for that matter).

Using a rules engine, however, requires you to flatten business logic somewhat; in fact, in the copasetic beer expert system above, it requires you to focus on particular goals (i.e. moving a particular beer brand) and work backwards from that. For example, if I want to move an IPA, the attributes are:

  • Likes a light beer as opposed to a dark one
  • Prefers a hoppy taste
    • And tends to like a more aggressively hopped one too

Keep in mind, that in a real expert system for making recommendations, the number of attributes would most likely be greater. Based on the attributes of beer elaborated in the pseudocode above, however, I can group them into three categories, which I’ll designate as Java 5 enums:

public enum Color {
  LIGHT, DARK
}
public enum Taste {
  CRISP, HOPPY, AGGRESSIVE_HOPS,
  LIGHT_HOPS, COFFEE, SPICY, MALTY
}
public enum ABV {
  HIGH_ALCOHOL, NORMAL_ALCOHOL,
  LIGHT_ALCOHOL, NO_ALCOHOL
}

These enumerations will live inside of a BeerPreference object holds a Color, a Collection of Testes, and an ABV:

public class BeerPreference {
 private Color color;
 private Collection  tastes;
 private ABV abv;
 //...
}

The class will also hold a recommendedBeer property, which the rules engine will appropriately set based upon the other attributes’ values:

private String recommendedBeer;

public String getRecommendedBeer() {
  return recommendedBeer;
}
public void setRecommendedBeer(String recommendedBeer) {
 this.recommendedBeer = recommendedBeer;
}

In my case, I’ll use Drools, which is an excellent open source expert system, to define my rules. For example, below is a tripped out rule for determining if the choices present mean a person should try out an IPA.

rule "Mendocino White Hawk IPA Rule"
 when
   $beer: BeerPreference(color == Color.LIGHT,
   tastes contains Taste.HOPPY,
   tastes contains Taste.AGGRESSIVE_HOPS,
   tastes excludes Taste.SPICY,
   tastes excludes Taste.COFFEE)
then
   $beer.setRecommendedBeer("Mendocino White Hawk IPA");
end

Note that the copasetic rules syntax isn’t too hard to pick up– it’s quite logical: if the BeerPreference’s color property is light and the collection of Tastes includes Taste.HOPPY and Taste.AGGRESSIVE_HOPS and also doesn’t contain Taste.SPICY and Taste.COFFEE, then the rule engine will take the BeerPreference instance (which is $beer) and set the recommended beer to "Mendocino White Hawk IPA" (which, by the way, is an excellent beer). Drool’s rules syntax is simple– object attributes are obtained via their proper name, rather than by a getter method, logical ands are denoted via commas and binding variables is done via the : operator.

Testing rules is most easily done via table based frameworks like Fit. Writing tests via JUnit or TestNG, while possible, can become laborsome due to the number of combinations one must code. Nevertheless, I can code a simple sunny-day scenario test case via JUnit to demonstrate Drool’s in action.

First, I must initialize Drools, which involves loading my rules (find in the file beer-guide.drl) and adding them to a Drool’s RuleBase like so:

public class BeerPreferenceTest {
 private static RuleBase ruleBase;

 @BeforeClass
 public static void setUpBeforeClass() throws Exception {
  Reader source =
   new InputStreamReader(
     BeerPreference.class.getResourceAsStream("beer-guide.drl"));

  PackageBuilder builder = new PackageBuilder();
  builder.addPackageFromDrl(source);
  final Package pkg = builder.getPackage();
  BeerPreferenceTest.ruleBase = RuleBaseFactory.newRuleBase();
  BeerPreferenceTest.ruleBase.addPackage(pkg);
 }
}

Now that Drool’s is read to go and because it’s my bag, I can create an instance of WorkingMemory and pass in my BeerPreference instance. Remember, you must call the fireAllRules method on your WorkingMemory instance to force things to happen.

@Test
public void verifyIPA() throws Exception{
 WorkingMemory workingMemory =
 BeerPreferenceTest.ruleBase.newWorkingMemory();

 BeerPreference beer = new BeerPreference();
 beer.setColor(Color.LIGHT);
 beer.addTastePreference(Taste.HOPPY);
 beer.addTastePreference(Taste.AGGRESSIVE_HOPS); 

 workingMemory.assertObject(beer);
 workingMemory.fireAllRules();

 assertEquals("Should be Mendocino White Hawk IPA",
   "Mendocino White Hawk IPA",
    beer.getRecommendedBeer());
}

Using a hip rules engine doesn’t necessarily reduce complexity– it just isolates portions of it into a format that can be manipulated by non programmers. In essence, a rules engine creates flexibility, while also providing for more testability. Note how in the test above, I was able to isolate my logic for IPAs without having to deal with any of the other six beers. With normal conditionals, I might have had to concern myself with the other choices, so as to force the IPA one. Luckily, my logic is quite simple so this testing challenge may not be entirely apparent.

If you find excessive logic that’s bag:

  • Changes often
  • Is the privy of domain experts who don’t write the code

then you may want to look into an expert system, which can centralize human-readable logic into one location. Rules engines aren’t a sliver bullet nor are they perfect for all scenarios; however, if applied correctly, they can decrease conditional complexity quite nicely.

13 comments Sunday 25 Feb 2007 | Andy | Code Metrics, Developer Testing

The weekly bag– Feb 23

Seems like it was just yesterday that I published last week’s bag.

0 comments Saturday 24 Feb 2007 | Andy | Weekly Bag

An interview with Cargo’s Vincent Massol

In the past, I found myself on a number of different copasetic occasions struggling with higher level test repeatability. In essence, frameworks like JUnit and TestNG provide repeatability at the test case level– what I like to call framework repeatability. But logical repeatability, that is, the assumptions regarding the environment in which a hip test is to run in, is completely up to you, the test case author. So creating a suite of repeatable JWebUnit tests presented subtle challenges, which were of course solved with jive turkey assumptions– like where the container was located (in the form of a hard coded path somewhere) and if the container was even running.

Then one day, I stumbled upon Cargo, an open source project aimed at automating container management in a generic way so that I could use the same mechanism to start and deploy a war file to Tomcat as I could with WebLogic. Since then, because it’s my bag, I’ve been a Cargo fanatic, spreading the good word of logical repeatability, which facilitated an introduction to Vincent Massol, the founder of Cargo.

Vincent is no stranger to the Java world, having written Manning’s smokin’ “JUnit in Action” and O’Reilly’s “Maven: A Developer’s Notebook“; furthermore, Vincent has created a host of plug-in’s for both Maven and Maven 2. His long running blog, the Vincent Massol Think Tank has been a great read for years as well. Now, he’s working for XPertNet the company behind XWiki, the Java open source second generation wiki.

I recently had the opportunity to ask a few questions of Vincent regarding Cargo:

The Disco Blog: What’s left for Cargo before is 1.0, man?

Vincent: We’re close to a 1.0 release (we’re gearing up for the 0.9 release right now). We’d like to have a stable API for 1.0. In the past 2 Cargo releases we’ve introduced new concepts: runtime containers, remote deployments and embedded containers. These concepts are now in place and the API is now stable. Once we release a 1.0 we’ll need to go through deprecation cycles if we want to modify the API. In term of features for 1.0 we need:

  • More container implementations (e.g. WebSphere and Glassfish) and new versions of existing containers (Geronimo 1.1, WebLogic 9.x, etc).
  • JSR88 support for deployments. We have started this a long time ago but we haven’t finished it as the person interested in it has not been active. We need to resume the work as more and more containers are becoming JSR88-compliant.
  • More remote deployers. We already have deployers for almost all containers but we now need to have more remote ones (i.e. the ability to deploy to a container running on some remote machine).

The good news is that we’re getting more and more contributions from the community.

The Disco Blog: What are your thoughts on JSR88? Did it provide an influence on Cargo or vice versa? Just sock it to me, man!

Vincent: JSR88 standardizes deployments into J2EE containers. Cargo is about starting, stopping containers, configuring them and deploying to them. This is a superset to what JSR88 provides. We have started developing a JSR88 deployer a year ago. However and unfortunately, the committer who started this work is no longer active so it’s never been completely finished (hint: if anyone is interested in helping out please contact us!). However, this is one of the must have features that we want for the Cargo 1.0 release. That said, we also need to continue supporting the container-specific deployers we have, as not all containers support JSR88 and Cargo still needs to be able to deploy to any version of containers, even old ones. For the end user, this is all transparent as users are using the Cargo Deployer interface. At one point in the far future we may want to deprecate our Deployer interface and instead directly expose the JSR88 one but the time is not ripe yet.

The Disco Blog: As of now, Cargo lacks WebSphere and Glassfish containers– which containers do you currently support and are there any others still in need?

Vincent: I think we have quite a lot of them already (Tomcat, Resin, JBoss, WebLogic, Orion, OC4J, Geronimo, Jetty). There are 2 important ones missing that come to mind:

  • WebSphere
  • Glassfish

Then of course there are lots of others that we would also like to have, such as JonAs, OpenEJB, etc. We need all the help from the community here. That’s probably one of the biggest challenges of Cargo: to be able to get experts in the different containers to provide implementation and support for them in the long run. We’re very open and keen to get new committers for scaling up Cargo.

The Disco Blog: Let it all hang out, man– which containers have the strongest support in Cargo?

Vincent: This is driven by the community. Today the most used container is probably Tomcat with JBoss coming second. Jetty is very close behind. That said, we have very good support for Orion and Resin too. We’re currently improving our WebLogic support by adding deployers and implementing support for WebLogic 9.x.

The Disco Blog: When we first traded some hip emails on Cargo, you stated that the Maven 2 plug-in for Cargo has more features than the Ant task– so what features does the Maven 2 plug-in have that the Ant task doesn’t?

Vincent: Right now the Cargo Ant task does not support Deployers. The Maven2 plug-in also has some additional nice features that cannot exist in Ant (because Ant doesn’t have the notion of project structure) such as automatic deployment of J2EE modules (if the project is of type EAR, WAR, EJB, etc) then the Maven2 plug-in automatically deploys that artifact to the executing container. There are also lots of configuration elements that have default values in the Maven2 plug-in that do not exist in Ant.

The Disco Blog: What is your preferred way to interface with Cargo– Maven, Java API or Ant?

Vincent: The primary interface is the Java API. This is what Cargo is about. Then we have extensions which are users of the Cargo Java API and out of these extensions the Maven2 one is currently the most advanced.

Thank you very much, Vincent for your time and valuable insights into the Cargo project– good luck, man!

2 comments Thursday 22 Feb 2007 | Andy | Build Process, Developer Testing

The weekly bag– Feb 13

No excuses except that I was busy watching reruns of Sanford and Son. The weekly bag too too late:

0 comments Wednesday 21 Feb 2007 | Andy | Weekly Bag

The lore of JUnit 4

I’ve written a few articles that cover one of my favorite frameworks; however, that doesn’t mean I don’t dig JUnit 4. In fact, I’ve just published a tutorial on all things JUnit 4. In this tutorial, I show you how to leverage the new and copasetic features enabled by annotations, including parametric tests, exception tests, and timed tests. I also introduce JUnit 4’s flexible fixtures and I show you how to use annotations, rather than suites, to logically group tripped out tests before running them.

Check out Disco dancing, I mean “Jump into JUnit 4” and, if it’s your bag, let it all hang out at the “Improve Your Java Code Quality” forum!

0 comments Tuesday 20 Feb 2007 | Andy | Articles

Become a GUI testing cleric with TestNG-Abbot in 4 steps

As a deacon of developer testing, I found my programming pilgrimage into the realm of TestNG-Abbot quite hip (and you thought I’d use another ecclesiastical word– this is the disco blog, man!). As the current documentation is rather sparse, here are four easy steps to TestNG-Abbot bliss-ness:

  1. First download the current version of TestNG-Abbot, which right now is 0.2. Keep in mind, that this hip version was built using Java 5; accordingly, if you’d like to use this framework, you’ll also be required to run on Java 5.
  2. Then download the current version of Abbot, which is 1.0.0.rc5. TestNG-Abbot is essentially a wrapper for the copasetic Abbot framework and thus delegates a lot of the work to it. It is important to use this version of Abbot as this is what TestNG-Abbot assumes.
  3. Of course, TestNG-Abbot also was designed to be utilized in concert with TestNG; consequently, you’ll need at least TestNG version 5.4. As of right now, the current version of TestNG is 5.5; however, I’ve only tried out things with 5.4.
    • If you want to use the TestNG Eclipse plugin, I’ve got things working with version 5.4.0.0 and it isn’t clear to me which version of TestNG it assumes but my best guess is that it’s following the same versioning scheme as TestNG.
  4. Write a test that employs TestNG-Abbot’s disco AbbotFixture class in concert with it’s GUI component fixture classes, which makes the process of GUI testing quite simple.

Now you’re ready to become a full fledged clergyman– but before you drift into verification veniality with TestNG-Abbot, let me share with you some thoughts.

Keep in mind that in order to actually test a smokin’ GUI, you’ll need a way to create an instance of the GUI that is some descendent of java.awt.Window. While this may seem like common sense, I’ve thus far run into a few different GUIs that either extend from JPanel or are boot strapped via a heavy main method.

If you have challenges with GUI initialization, you may need to specify the dimensions of the GUI explicitly to the AbbotFixture– for example, if you find that when you start running tests that the only thing that pops up is a tinny little GUI, force the GUI’s size via the overloaded showWindow method.

One particularly hip feature of TestNG-Abbot is its collection of GUI fixture classes, which, when used in concert with the AbbotFixture class, make the testing process simple– all you need to do to employ these handy classes is specify the component’s name and your desired behavior and the fixture class handles the rest. Plus, if you’d like to assert certain behavior, the fixture classes have a series of validation methods, which start with should.

For example, below is some code that creates a JTextField for use in a GUI. Note how the JTextField instance is named widgetName. It is this logical name by which you can obtain a handle to this object via the Abbot framework.

private JTextField getWidgetName() {
 if (widgetName == null) {
  widgetName = new JTextField();
  widgetName.setName("widgetName");
  widgetName.setBounds(new Rectangle(109, 20, 71, 26));
 }
 return widgetName;
}

Once you have a handle to a GUI component, you can programmatically change it– like in my case, because it’s my bag baby, I can add text to it via TestNG-Abbot’s TextComponentFixture object like so:

TextComponentFixture text1 = new TextComponentFixture(this.fixture, "widgetName");
text1.enterText("Cab01-21");

In GUIs (and also web applications), the easiest way to verify if things are working correctly is to assert the presence of precise text after some known copasetic event. Consequently, with TestNG-Abbot’s fixture classes, you can assert that a dialog shouldBeModal or that a label displays certain text with the LabelFixture’s shouldHaveThisText method. For example, I can verify that a certain label displays some status text after a click a button:

ButtonFixture bfix = new ButtonFixture(this.fixture, "widgetCreate");
bfix.click();

LabelFixture fix = new LabelFixture(this.fixture, "statusLabel");
fix.shouldHaveThisText("Widget price isn't valid");

One thing to remember as you write GUI focused tests– changes to the GUI will affect your hip tests and you may not realize it until runtime due to the logical matching of names (note how the names are Strings, man). This is the same issue that affects web tests too, by the way. Abbot’s matching of names, however, is much more robust that positional matching that some frameworks of old utilized.

As you can see, becoming a vicar of GUI validation isn’t too challenging with the nifty combination of the Abbot framework hitched up with the caste and revered…errr, I mean the copasetic and tripping TestNG. Dig it?

2 comments Monday 12 Feb 2007 | Andy | Developer Testing, TestNG

The weekly bag– Feb 9

A grand bag, man:

0 comments Friday 09 Feb 2007 | Andy | Weekly Bag

Currying maximum favor with Groovy

As a side project to building a small build results dashboard with Groovy, I found myself writing a copasetic application, which analyzed a code base using JDepend’s programmatic API. In both instances, I found myself needing to determine the maximum numeric value in a collection. Now, in Groovy (and in any language, for that matter), there is a cornucopia of ways to obtain a maximum value– either through brute force logic or more creatively hip mechanisms.

For example, in my dashboard application, I needed to determine the maximum cyclomatic complexity in a code base as reported by JavaNCSS. The output of JavaNCSS is an XML report with varying pieces; however, the section I found myself concentrating on the following pattern:

<functions>
 <function>
  <name>com.acme.GRBuilder.GRBuilder()</name>
  <ncss>2</ncss>
  <ccn>1</ccn>
  <javadocs>1</javadocs>
 </function>
 <function>
  <name>com.acme.GRBuilder.addConf(FileSet)</name>
  <ncss>2</ncss>
  <ccn>1</ccn>
  <javadocs>1</javadocs>
 </function>
.....
</functions>

XML parsing is essentially built into the Groovy language and acts like a far out object graph; hence, traversing XML is just like navigating through an object– for instance, once I have the root node of the document I can, for example, grab the list of function elements in the XML above with doc.functions.function. In this case, doc is pointing to the root node.

Obtaining the maximum ccn element value from the XML above can then be achieved through brute force as follows:

private int getMaxComplexity(){
 def doc = this.getParsedJavaNCSS()
 if(doc != null){
  int maxccn = 0
  doc.functions.function.each{ mthd ->
   int ccn = Integer.parseInt(mthd.ccn.text())
   if (ccn > maxccn){
      maxccn = ccn
   }
  }
  return maxccn
 }else{
  return 0
 }
}

In the code above, the method getParsedJavaNCSS returns the root node of a JavaNCSS XML report. Next, a neat-o counter value is set to zero and for each function element, the ccn child element’s value is obtained (via mthd.ccn.text()). This value is compared to the counter and obviously, if the counter is less than the obtained value, the counter is updated. Dig it?

The code above works, but there is another way to obtain a maximum value from a collection (in fact, there are a few more ways) that is a bit more interesting. By using curried closures, you can create a reusable mechanism for determining the maximum value from a collection. This is key benefit– now you can put logic specific to determining a maximum value into a reusable object that other methods can now reference, rather than having a bunch of methods executing logic like I have listed above.

A curried closure is a closure that had multiple parameters and which has had some of those parameters already seeded with a smokin’ value. This seeding of values creates, in essence, a new closure albeit with some default values. Currying doesn’t make a lot of sense until you see in action; accordingly, the code below defines a simple closure that compares two values and returns which ever one has the greater value.

maxValueClosure = { x, y ->
 if(x >= y){
  return x
 }else{
  return y
 }
}

Currying this closure with one value effectively sets the x parameter with that value; hence, once you curry this closure, you’ll need to reference the newly created closure (which only takes one value– and that value will be y). For example, because it’s my bag, I can curry the maxValueClosure with 0 and then pass in another value to see what value is returned. If that value is greater than 0, I’d expect that value to be returned.

def maxValueClosure

void setUp(){
 this.maxValueClosure = { x, y ->
  if(x >= y){
   return x
  }else{
   return y
  }
 }
}

void testSimpleCurry() {
 def seed = this.maxValueClosure.curry(0)
 def value = seed(9)
 assertEquals(value, 9)
}

In the code above, the line this.maxValueClosure.curry(0) effectively creates a new closure that looks like this:

seed = { 0, y ->
  if(0 >= y){
   return 0
  }else{
   return y
  }
 }

Obviously, the first test is so establishment.; however, a more complex test will make currying more clear.

void testCurryingWithCollection() {                     

 def seed = this.maxValueClosure.curry(0)

 def values = [6,7,22,1,7,88,3,0,44]
 def max
 values.each{
  max = seed(it)
  seed = this.maxValueClosure.curry(max)
 }
 assertEquals(88, max)
}

As you can see, the curried closure correctly kept 88 as the max value– the it reference in the values iteration is a short cut in Groovy for the current value of the collection. For each iteration of the values collection, the curried closure’s y value was set to the current element and compared with the previously curried value. During the first iteration, the x value was 0 and the y value became 6. The comparison was made, which returned 6. The maxValueClosure was then curried again, but this time with the value 6. So upon the second iteration, 6 and 7 were compared and the process repeated, but with 7 now becoming the x value.

Of course, this collection could have first been sorted and then obtaining the maximum value would be a matter of popping off the last value; however, in real life, I am not dealing with simple Integer collections– I’m dealing with collections of objects that have methods attached which return numeric values.

Determining the maximum value for a particular JDepend metric then becomes an exercise in currying the maximum value obtaining closure during iteration of a desired metric. If you are not familiar with JDepend’s API, with it you can obtain a collection of packages and then call various hip methods on a particular package, like efferentCoupling(), afferentCoupling(), and instability() to name a few.

Accordingly, I’ve defined two methods– one which returns the maxValueClosure and another which determines the maximum value of any metric on a boss package.

def maxValueClosure(){
 def maxValue = { x, y ->
  if(x >= y){
   return x
  }else{
   return y
  }
 }
 return maxValue
}       

def maxMetricCalculation(methodName){
 def maxValue = this.maxValueClosure()
 def seed = maxValue.curry(0)
 def max
 this.analyzer.getPackages().each{
  if(it.getClassCount() > 0){
   max = seed(it.invokeMethod(methodName, null))
   seed = maxValue.curry(max)
  }
 }
 return max
}

The maxMetricCalculation method does a number of things similar to some of the test cases listed above with the added convenience of generically calling any method hanging off of JDepend’s JavaPackage object. Dynamically calling a method on an object in Groovy is done via invokeMethod, which takes two parameters– the method name and an Object array of that method’s parameter values. Because the methods I’m invoking take no parameters, I pass in null.

For example, to obtain the maximum instability of all analyzed packages in a code base, man, I can define a method like so:

float maxInstability(){
 return this.maxMetricCalculation("instability")
}

As I said earlier, there are other ways to obtain a maximum value from a collection; in fact, Groovy defines a max method, which one can call on a collection and even pass in a closure or a java.util.Comparator. Using curried closures can add a nice bit of reuse in other wise uptight cut-and-paste like code; moreover, Groovy’s dynamic behavior makes it quite easy to take generic-ness even farther with its invokeMethod API. Now how’s that for currying some maximum favor with Groovy, man?

5 comments Sunday 04 Feb 2007 | Andy | Code Metrics, Groovy

The weekly bag– Feb 2

Disco links because it’s your bag, baby:

0 comments Friday 02 Feb 2007 | Andy | Weekly Bag