The Disco Blog Welcome Guest | Login or Register

All things related to code quality, man! This week’s bag includes:

• The World’s Simplest Unit Testing Framework- Cay Horstmann describes using println’s for assertions.
• Unit Testing Tips- Cameron Purdy lays out 6 hip tips for effective developer testing.
• Why Do Pointless if Statements Even Compile?- Many a nefarious defect has arisen from this relaxed syntactical rule of the Java language.
• Testing J2ME applications- Yours truly points out some interesting J2ME testing articles.
• How JavaNCSS calculates cylcometric complexity- rwhansen muses about the differences between how various tools calculate cyclomatic complexity.
• Is Test-Driven Development Overrated?- Adam Wiggins doesn’t think so, man!
• Code Coverage Coverage- Mandy Ownes has some good vibes on code coverage.
• moreUnit, a handy Eclipse plug-in for marking up tested code and generated test stubs version 0.6.1 was released
• You’re ugly and your code is, too- Burke Cox doesn’t miss a beat to tell you how he feels.
• Managed Debugging Assistants- Managed Debugging Assistants work with the CLR to provide information on runtime state. Neat-o!
• 2007 Conference for the Association of Software Testing- call for papers open.
• AnthillPro 3.0 Released- The boys in Cleveland have an interesting Build Management Server.
• Java Unit Testing and Refactoring Event!- Codign Software and Vision Technology Services are hosting a tripping free workshop on Java unit testing and refactoring in Baltimore.

moreUnit is a copasetic Eclipse plug-in that provides two interesting features:

1. It marks up classes, which have a corresponding developer test
2. It generates a test case stub for desired methods

The tool is heavily focused on JUnit style testing; however, moreUnit works for TestNG as well. As I’ve already advocated, TestNG and JUnit play nicely together, so it shouldn’t be a surprise to see moreUnit complement TestNG.

moreUnit discovers corresponding tests via naming patterns; hence, if you have a hip class named Hierarchy, it’ll scan a project for a HierarchyTest or a TestHierarchy class. Because it’s my bag, I have quite a few code bases that have both TestNG and JUnit tests, I tend to specify TestNG tests via the TestNG suffix and leave JUnit tests with the traditional Test suffix.

Accordingly, I just had to add the new suffix to the moreUnit preference dialog shown below.

If you already have a TestNG test defined for a class (and moreUnit has spotted it), you can right click on non-tested methods and test cases will be generated. Just make sure you’ve unclicked “Create JUnit4 methods” option– otherwise you’ll have to fix an errant JUnit import. With the option unclicked, you’ll be left with a tripping old style JUnit test case (returning void and starting with test).

As you can see in the above figure, you can either right click over an untested method or type Ctrl+U to generate a test case stub.

All that’s left after the stub is created is an annotation (1.5 or 1.4 JavaDoc) and of course, the test logic. Dig it?

Looking for an effective Continuous Integration feedback mechanism, that isn’t email? Check out Yahoo! Widgets, which are JavaScript applications that run outside of a browser and run on both Windows and Mac OS X. If you haven’t seen them yet, check them out as they are indubitably hip.

There are three Widgets currently available, which can hook into CruiseControl (two of which also support CruiseControl.NET):

• CruiseControl Dashboard (Java only, but displays an ever so groovy Lava Lamp)
• Cruise Control Status
• The generic Cruise Control widget

These applications offer similar features to Window’s tray icon, but are a bit more aesthetic. Outta sight, man!

Cylcomatic complexity precisely measures complexity by counting the distinct paths through a hip method. Various studies over the years have determined that methods having a cyclomatic complexity (or CC) greater than 10 have a higher risk of defects. Consequently, monitoring CC is a groovy way to keep code entropy at bay– if you see complexity increasing, you can either:

• Write additional tests that focus on the complexity (in which case, you’d need an equal number of test cases to CC to reach 100 percent coverage of a method)
• Refactor the method into smaller methods, which can then be tested more easily

The reason why CC is an excellent gauge for complexity stems from the notion that the average person can handle about seven disco items in their mind (plus or minus three) at any given point in time. Hence, for most people, phone numbers are easier to memorize than credit card numbers (and for that matter, disco dance moves requiring more than seven steps).

The same notion applies to code– comprehending a method that has a lot of paths (via conditionals, for example) becomes increasingly challenging as the number of paths accumulates. Accordingly, when a developer becomes intellectually challenged (say, when modifying a nefariously long method), a defect is most assuredly waiting to manifest itself, man.

Because pathing complexity affects humans and not machines, CC is a source code metric– it’s only meaningful in the context of readable code that someone has to maintain. Compiled code is a different looking beast; in fact, most compilers will optimize code to reduce far out paths. Consequently, binary CC will most always be lower than source code CC. In most cases the values won’t be terribly different; however, if you find yourself using tools that determine CC by analyzing byte code, be mindful that the values they report may not line up with what you see. This could lead to a false sense of security.

In the Java world, Cobertura reports CC obtained from byte code. Additionally, the tool appears to “average McCabe’s cyclomatic code complexity for all methods.” Averaging CC somewhat helpful, however, you’ll find that CC is most copasetic in the context of a single method.

In the .NET world, CCMetrics also reports CC from byte code. In fact, most .NET code analyzers I’ve used examine binaries.

The next time you see a CC value, take a moment to understand how the value was calculated and always double check the offending source code. While metrics are objective, their application should be subjective. Dig it?

During a hip presentation in Boston, I was reminded that one can simulate test groups via JUnit’s test suite mechanism. While I tend not to use suites due to copacetic runners in Ant and Maven, the technique is certainly applicable and for some, it’s more convenient. Plus, by using test suites, one doesn’t need to worry about naming patterns or directory structures (that is, until they need a way to segregate the suites themselves). On the downside, of course, the problem with suites is that they are inherently a manual process– one must add new tests to them pragmatically.

For example, I can define a suite of component tests as follows:

package test.org.acme.widget;

import junit.framework.Test;
import junit.framework.TestSuite;
import test.org.acme.widget.dao.*;

public class ComponentTestSuite {

 public static void main(String[] args){
  junit.textui.TestRunner.run(ComponentTestSuite.suite());
 }

 public static Test suite(){
  TestSuite suite = new TestSuite();
  suite.addTestSuite(SpringWidgetDAOImplTest.class);
  suite.addTestSuite(WidgetFinderTest.class);
  suite.addTestSuite(UserWidgetTest.class);
  suite.addTestSuite(WidgetAccessTest.class);
  return suite;
 }
}

The ComponentTestSuite class defines a suite method, which will reflectively add all tests in the four test classes added in the addTestSuite methods. Therefore, when this class is executed (either via Ant, Maven, the command line, or even TestNG) all component tests are run. Note, if a new component test class is defined, it must be added in this class’s suite method (which requires a recompile).

Interestingly, the JUnit-addons and GSBase projects have some dynamic APIs, which facilitate auto-discovering JUnit tests in directories; however, because it’s my bag, I’d argue that you may be better served by following a directory naming pattern and relying on Ant’s junit task to do the same. Do you dig it, man?

Over on ontoinfo.com, an interesting article was published entitled “Making Software Quality Metrics Actually Work” in which the author states that “No one quality metric can provide [an] accurate and complete picture.” Bingo! They hit the nail on the head with that one, man.

A metric, in and of itself, is not very useful– for example, code coverage is simply a number that, in isolation, doesn’t mean anything. Highly covered code is still susceptible to defects! Coverage also is not an indication of test quality, by the way. Coverage is simply a measure of test execution– what’s important is how the coverage values are utilized and compared against other hip metrics or trends.

The obvious immediate value of coverage reports is not in what has been tested, but in what’s not tested. Monitoring this value, in concert with complexity, is a valuable strategy that enables teams to quickly ascertain areas of associated risk (i.e. areas of code with high complexity and low code coverage should be a focus point– either through additional developer testing or even coordination with QA efforts).

Maintenance metrics is another grouping of copasetic metrics that work well in concert. Coupling alone is not all that helpful– it’s when these metrics are combined with Abstractness or Instability or even code coverage that a window into risk can be viewed. And only then, when metrics are viewed in relation to other metrics or even when viewed over time can you actually begin to make metrics work. Dig it?

As I wrote about previously, TestNG can execute an existing suite of JUnit tests, thus providing a unified reporting mechanism for all developer tests. This means that TestNG can also execute developer test written in Groovy.

In order for this process to work, however, you must compile any Groovy tests into byte code and add references in TestNG’s testng.xml file. You can either:

• Reference the corresponding class directly or
• Reference the package the class resides in

and ensure that the junit attribute of the test element these classes are defined in is set to true.

The following test, for instance, defined in a testng.xml file, is designated as a JUnit test. It will be run JUnit style– a fixture (if defined) for every test found in the class.

<test name="junits" junit="true">
  <classes>
   <class name="test.com.acme.sedo.frmwrk.filter.impl.ClassInclusionTest" />
  </classes>
</test>

In order to run all copasetic unit tests (defined in both TestNG and Groovy) in an Ant build, I create a TestNG task, which depends on a compilation of Groovy files, shown below.

<groovyc destdir="${classesdir}"
    srcdir="./test/groovy" >
   <classpath>
    <path refid="build.classpath"/>
    <pathelement path="${testclassesdir}"/>
    <pathelement path="${classesdir}"/>
   </classpath>
</groovyc>

The task above compiles any associated Groovy tests (found in the test/groovy directory) and puts the class files in a specific location.

Now, when testng-unit is invoked, any corresponding hip tests written in Groovy will be compiled into normal .class files and therefore picked up by TestNG (and run as normal JUnit tests). The TestNG task is accordingly defined to pick up a testng.xml file designated for running unit tests.

<target name="testng-unit" depends="compile,groovy-compile">
 <testng outputDir="${testng.output.dir.unit}"
        sourceDir="${testng.source.dir}"
        classpath="${testclassesdir};${classesdir}">

   <xmlfileset dir="${testng.suitexml.dir}"
        includes="${testng.suitexml-unit.name}"/>

   <classpath>
     <path refid="build.classpath"/>
   </classpath>
  </testng>
</target>

Now that this process has been defined, I can write tests in either JUnit, TestNG, or even Groovy and one runner will run them all. Dig it, man?

TestNG’s flexible fixtures, parametric testing, and hip grouping facilities make higher level testing a breeze, yet, sometimes I find myself still writing basic unit tests with JUnit (old habits never die, I suppose). But, as I’ve stated before, there’s nothing wrong with using the two smokin’ frameworks in concert– in fact, many code bases already have an extensive collection of JUnit tests; consequently, TestNG can run JUnit tests.

Using TestNG’s testng.xml suite file, you can specify a collection of JUnit tests by creating a new test entry and setting the junit attribute to true. For example, the following suite file runs a series of TestNG tests as well as all the JUnit tests found in the test.com.acme.pdp and test.com.acme.pdp.gsap packages.

<!DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd">
<suite name="all unit tests suite">
<test name="all unit tests">
 <groups>
  <run>
   <include name="unit"/>
  </run>
 </groups>
 <packages>
  <package name="test.com.acme.pdp.ng"/>
 </packages>
</test>  

<test name="junits" junit="true">
 <packages>
  <package name="test.com.acme.pdp"/>
  <package name="test.com.acme.pdp.gsap"/>
 </packages>
</test>
</suite>

As you can see, this suite file defines a group of TestNG tests that focus on unit tests and another group of JUnit tests found in two different packages.

TestNG isn’t an all-or-nothing choice when it comes to using a testing framework. If you already have an existing suite of copasetic JUnit tests, you can still run them via an Ant task; however, running them via TestNG provides one single report for test results.