The Disco Blog Welcome Guest | Login or Register

Studies have shown that the average person has the capacity to handle about seven pieces of data in their head, plus or minus two [ref]. Hence, we can easily remember phone numbers, but most of us have a slightly more difficult time memorizing credit card numbers, groovy bellbottom widths, launch sequences, boss disco moves, etc.

This principle applies to understanding code. Everyone has probably seen snippets of code like this:

if (entityImplVO != null) {
  List actions = entityImplVO.getEntities();
  if (actions == null) {
     actions = new ArrayList();
  }
  Iterator enItr = actions.iterator();
  while (enItr.hasNext()) {
    entityResultValueObject arVO = (entityResultValueObject) actionItr
     .next();
    Float entityResult = arVO.getActionResultID();
    if (assocPersonEventList.contains(actionResult)) {
      assocPersonFlag = true;
    }
    if (arVL.getByName(
      AppConstants.ENTITY_RESULT_DENIAL_OF_SERVICE)
         .getID().equals(entityResult)) {
      if (actionBasisId.equals(actionImplVO.getActionBasisID())) {
        assocFlag = true;
      }
    }
    if (arVL.getByName(
     AppConstants.ENTITY_RESULT_INVOL_SERVICE)
      .getID().equals(entityResult)) {
     if (!reasonId.equals(arVO.getStatusReasonID())) {
       assocFlag = true;
     }
   }
 }
}else{
  entityImplVO = oldEntityImplVO;
}

There are arguably 9 different paths shown there. Incidentally, this snippet is part of a larger 350 plus line method, which was shown to have 41 distinct paths. Imagine if you were tasked to modify this method for the purposes of adding a new feature. If you didn’t write this method, do you think you could make the requisite changes without introducing a defect? Keep on truckin’, man.

Of course, you’d write a test case, but do you think your test case could isolate your particular change in that sea of conditionals?

Cyclomatic complexity, which was pioneered in the Age of Disco (!!), precisely measures this path complexity. By counting the distinct paths through a method, this integer based metric aptly depicts method complexity. In fact, various studies over the years have determined that methods having a cyclomatic complexity (or CC) greater than 10 have a higher risk of defects [ref].

Because CC represents the paths through a method, this is an excellent number for determining the number of test cases required to reach 100% coverage of a method. For example, the following not-so-copasetic method has a logical defect.

public class PathCoverage {
  public String pathExample(boolean condition){
     String value = null;
      if(condition){
          value = " " + condition + " ";
      }
      return value.trim();
   }
}

One test can be written which, interestingly enough, achieves 100% line coverage.

import junit.framework.TestCase;

public class PathCoverageTest extends TestCase {
    public final void testPathExample() {
    	PathCoverage clzzUnderTst = new PathCoverage();
        String value = clzzUnderTst.pathExample(true);
        assertEquals("should be true", "true", value);
    }
}

Running a code coverage tool, such as Cobertura yields the following report:

This method has a CC of 2 (one for the default path and one for the if path). Using CC as a more precise gauge of coverage implies a second test case is required. In this case, it would be the path taken by not going into the if condition as shown below.

public final void testPathExampleFalse() {
   PathCoverage clzzUnderTst = new PathCoverage();
   String value = clzzUnderTst.pathExample(false);
   assertEquals("should be false", "false", value);
}

Of course, running this new test case yields that nasty NullPointerException.

Luckily, the method under test in this case only has a CC of two. Imagine if that defect was buried in a method with a CC of 102. Good luck finding it, man. Unfortunately, I routinely run across code with CCs in the 100s too.

Because CC is such a good indicator of code complexity, there is a strong relationship between test driven development and low CC values. When tests are written often (note, I’m not implying first), developers have the tendency to write uncomplicated code because complicated code is hard to test. If one finds they are having difficulty writing a test it’s a red flag that the code under test may be complex. The short code, test, code, test cycle invites refactoring in these cases, which continually drives the development of un-complex code.

By determining the CC of class methods in a code base and continually monitoring these values, development teams can keep tabs on code complexity and take appropriate actions to address complexity issues as they arise. Dig it?

For more information on CC and how to refactor high CC code, check out OnJava’s Code Improvement Through Cyclomatic Complexity (written by yours truly).

Many web applications work against databases, man. Databases, however, present quite a large dependency for testing, leaving one with two choices- either mock out as much as possible and avoid the database all together for as long as possible or suck it up and utilize the database. The latter choice presents a new series of challenges- how does one control the database during testing? Even better, how does one make those tests repeatable?

By far the easiest way to make your testing cake and eat it is to use a database seeding framework like any of the xDbUnits. These hip frameworks abstract a database’s dataset into XML files and then offer the developer fine grained control as to how this data is seeded into a database during testing.

For example, the following snippet is from a DbUnit XML seed file.

 <word WORD_ID="1" SPELLING="pugnacious" PART_OF_SPEECH="Adjective"/>
 <definition DEFINITION_ID="10"
   DEFINITION="Combative in nature; belligerent."
   WORD_ID="1"
   EXAMPLE_SENTENCE="The pugnacious youth had no friends left to pick on."/>
 <synonym SYNONYM_ID="20" WORD_ID="1" SPELLING="belligerent"/>
 <synonym SYNONYM_ID="21" WORD_ID="1" SPELLING="aggressive"/>  

Via DbUnit’s DatabaseTestCase, the data in the XML file is manipulated via operations such as insert, update, and delete. The specific database is configured by implementing the abstract getConnection method and the XML file is located via the getDataSet method. You can dig it below.

package test.org.aglover.words.dao;

import java.io.File;
import java.sql.Connection;
import java.sql.DriverManager;
import java.util.Iterator;
import junit.framework.TestCase;
import org.aglover.words.bizobj.IDefinition;
import org.aglover.words.bizobj.IWord;
import org.aglover.words.dao.exception.FindException;
import org.aglover.words.dao.impl.WordDAOImpl;
import org.dbunit.DatabaseTestCase;
import org.dbunit.database.DatabaseConnection;
import org.dbunit.database.IDatabaseConnection;
import org.dbunit.dataset.IDataSet;
import org.dbunit.dataset.xml.FlatXmlDataSet;

public class DefaultWordDAOImplTest extends DatabaseTestCase {
    /**
     * @see org.dbunit.DatabaseTestCase#getDataSet()
     */
    protected IDataSet getDataSet() throws Exception {
        return new FlatXmlDataSet(
          new File("test/conf/words-seed.xml"));
    }
    /**
     * @see org.dbunit.DatabaseTestCase#getConnection()
     */
    protected IDatabaseConnection getConnection() throws Exception {
        final Class driverClass =
              Class.forName("org.gjt.mm.mysql.Driver");

        final Connection jdbcConnection =
       	 DriverManager.getConnection(
              "jdbc:mysql://localhost/words",
       	      "words", "words");
         return new DatabaseConnection(jdbcConnection);
    }

    public void testFindVerifyDefinition() throws Exception{
       final WordDAOImpl dao = new WordDAOImpl();
       final IWord wrd = dao.findWord("pugnacious");

       for(Iterator iter = wrd.getDefinitions().iterator(); iter.hasNext();){
         IDefinition def = (IDefinition)iter.next();
         assertEquals("Combative in nature; belligerent.",
              "Combative in nature; belligerent.",
               def.getDefinition());
       }
    }  

    public DefaultWordDAOImplTest(String arg0) {
        super(arg0);
    }
}

Note though, that this trippin’ class makes the assumption that the database is located on the same machine that the test is run. This maybe a safe assumption on the desktop, but obviously this configuration can present a challenge in CI environments.

One solution is to pull out the hard coded connection strings and place them into properties files. There is however, a more effective mechanism. If DbUnit is utilized to seed a database, one can infer that the application itself then uses a database. If this is the case, it is a common best practice to avoid hard coding connection information within a code base; therefore, why not configure DbUnit to read the same file that the application under test reads? Far out!

For example, in Hibernate applications, database connection information is usually defined in the hibernate.cfg.xml file. One can easily write a utility class which parses this file and obtains the proper connection information. Even better, one can rely on Hibernate to provide the desired information as shown below.

//imports left out

public class DBUnitHibernateConfigurator {
 static Configuration configuration = null;

 private DBUnitHibernateConfigurator() {
  super();
 }

 private static Configuration getConfiguration()
	throws HibernateException {
  if (configuration == null) {
   configuration = new Configuration().configure();
  }
  return configuration;
}

public static IDataSet getDataSet(final String fileName)
    throws ResourceNotFoundException,
     DBUnitHibernateConfigurationException {
 try{
  return DBUnitConfigurator.getDataSet(fileName);
 }catch(DBUnitConfigurationException e2){
   throw new DBUnitHibernateConfigurationException(
      "DBUnitConfigurationException in getDataSet", e2);
 }
}

 private static String getProperty(final String name)
    throws HibernateException {
  return getConfiguration().getProperty(name);
 }

public static Properties getHibernateProperties()
    throws ResourceNotFoundException,
      DBUnitHibernateConfigurationException{
 try{
  final Properties hProp = new Properties();
  hProp.put("hibernate.connection.driver_class",
    DBUnitHibernateConfigurator.getProperty(
      "hibernate.connection.driver_class"));
  hProp.put("hibernate.connection.url",
    DBUnitHibernateConfigurator.getProperty(
      "hibernate.connection.url"));
  hProp.put("hibernate.connection.username",
    DBUnitHibernateConfigurator.getProperty(
      "hibernate.connection.username"));
  hProp.put("hibernate.connection.password",
    DBUnitHibernateConfigurator.getProperty(
      "hibernate.connection.password"));
  return hProp;
 }catch(HibernateException e){
   throw new DBUnitHibernateConfigurationException(
     "HibernateException in getHibernatePropertiesFile", e);
 }
}

 public static IDatabaseConnection getDBUnitConnection()
    throws DBUnitHibernateConfigurationException{
  try{
   final Properties props =
     DBUnitHibernateConfigurator.getHibernateProperties();
   return DBUnitConfigurator.getDBUnitConnection(props);
  }catch(DBUnitConfigurationException e1){
   throw new DBUnitHibernateConfigurationException(
     "DBUnitConfigurationException in getDBUnitConnection", e1);
  }catch (ResourceNotFoundException e2) {
   throw new DBUnitHibernateConfigurationException(
     "ResourceNotFoundException in getDBUnitConnection", e2);
  }
 }
}

Note how the above class puts the Hibernate connection info in a Properties object, which is then converted into DbUnit’s IDatabaseConnection type in a DBUnitConfigurator class. The DbUnit connection type is then returned via the getDBUnitConnection method. DbUnit’s IDataSet type, which represents those XML files containing all the data is returned via the getDataSet method. This copasetic method frees developers from having to provide a path to a file- something that can be tricky in different environments.

Next, a custom abstract test case class can be created which requests that implementers feed the desired dataset information for a particular test case.

//left out imports...

public abstract class DefaultDBUnitHibernateTestCase extends DatabaseTestCase {
 public DefaultDBUnitHibernateTestCase(String arg0) {
  super(arg0);
 }	

  protected void setUp() throws Exception {
   super.setUp();
   DefaultHibernateSessionFactory.
      closeSessionAndEvictCache();
   DefaultHibernateSessionFactory.
      getInstance().getHibernateSession();
  }

  protected void tearDown() throws Exception {
   DefaultHibernateSessionFactory.
      closeSessionAndEvictCache();
   super.tearDown();
 }

  protected IDatabaseConnection getConnection() throws Exception {
   return DBUnitHibernateConfigurator.
      getDBUnitConnection();
  }

  protected IDataSet getDataSet() throws Exception {
   final String fileName = this.getDBUnitDataSetFileForSetUp();
   DatabaseTestCase.assertNotNull("data set file was null", fileName);
   return DBUnitHibernateConfigurator.getDataSet(fileName);
  }

  protected abstract String getDBUnitDataSetFileForSetUp();
}

An example resulting test case which implements DefaultDBUnitHibernateTestCase is shown below.

//imports left out 

public class WordDAOImplTest extends DefaultDBUnitHibernateTestCase {

 public void testUpdateWordSpelling() throws Exception{
  WordDAOImpl dao = new WordDAOImpl();
  IWord wrd = dao.findWord("pugnacious");

  wrd.setSpelling("pugnacious-ness");
  dao.updateWord(wrd);

  IWord wrd2 = dao.findWord("pugnacious-ness");
  assertEquals("should be id of 1", 1, wrd2.getId());
 }

 public void testFindVerifyDefinitionsSize() throws Exception{
  WordDAOImpl dao = new WordDAOImpl();
  IWord wrd = dao.findWord("pugnacious");

  Set defs = wrd.getDefinitions();
  assertEquals("size should be one", 1, defs.size());
 }

 protected String getDBUnitDataSetFileForSetUp() {
  return "words-seed.xml";
 }

 public WordDAOImplTest(String arg0) {
  super(arg0);
 }
}

DbUnit offers an API (as seen above), which can be utilized effectively via composition- creating enormous opportunities for powerful combination frameworks too. With this added flexibility, testing various architectures at different layers in the Age of Aquarius becomes quite easy.

For example, developer testing of Struts applications can be challenging. A common tactic is to utilize a framework like HttpUnit, which simulates http requests; however, this can be tedious work and doesn’t offer the precision one may desire in a Struts architecture which heavily utilizes Action classes and a configuration for mapping requests.

The StrutsTestCase project was created to address this issue- with this framework one can easily isolate and test Struts’ Action classes. This project, however, requires a developer extend a base class which handles mocking of a servlet container. If a Struts application requires the use of a database you may be left in a quandary.

Via DbUnit’s API, a combination framework can be created which utilizes the seeding capabilities of DbUnit with the mocking capabilities of the StrutsTestCase project.

package com.vanward.resource.unittest.dbunit.struts;

import java.util.Properties;
import org.dbunit.DatabaseTestCase;
import org.dbunit.database.IDatabaseConnection;
import org.dbunit.dataset.IDataSet;
import org.dbunit.operation.DatabaseOperation;
import servletunit.struts.MockStrutsTestCase;
import com.vanward.resource.unittest.dbunit.\
   util.DBUnitConfigurator;

public abstract class DefaultDBUnitMockStrutsTestCase
   extends MockStrutsTestCase {

 public DefaultDBUnitMockStrutsTestCase(String testName) {
  super(testName);
 }

 public void setUp() throws Exception {
  super.setUp();
  this.executeOperation(this.getSetUpOperation());
 }

 public void tearDown() throws Exception{
  super.tearDown();
  this.executeOperation(this.getTearDownOperation());
 }

 private void executeOperation(DatabaseOperation operation)
    throws Exception{
  if (operation != DatabaseOperation.NONE){
   final IDatabaseConnection connection =
     this.getConnection();
  try{
   operation.execute(connection, this.getDataSet());
  }finally{
   closeConnection(connection);
  }
 }
}

 protected void closeConnection(IDatabaseConnection connection)
   throws Exception{
  connection.close();
 }

 protected abstract Properties getConnectionProperties();

 protected abstract String getDBUnitDataSetFileForSetUp();

 protected IDatabaseConnection getConnection() throws Exception {
  final Properties dbPrps = this.getConnectionProperties();
  DatabaseTestCase.
    assertNotNull("database properties were null", dbPrps);
  return DBUnitConfigurator.getDBUnitConnection(dbPrps);
 }

 protected DatabaseOperation getSetUpOperation() throws Exception {
  return DatabaseOperation.CLEAN_INSERT;
 }

 protected DatabaseOperation getTearDownOperation() throws Exception {
  return DatabaseOperation.NONE;
 }

 protected IDataSet getDataSet() throws Exception {
  final String fileName = this.getDBUnitDataSetFileForSetUp();
  DatabaseTestCase.assertNotNull("data set file was null", fileName);
  return DBUnitConfigurator.getDataSet(fileName);
 }
}

One again, you may be left with the option of hardcoding connection information or reusing existing files for this purpose! Testing a Struts application that uses Hibernate? Not a problem- just combine the new DefaultDBUnitMockStrutsTestCase with a handy dandy utility for reading Hibernate files.

For example, below is a class which implements a DefaultMerlinMockStrutsTestCase class, which combines the DbUnit ability of DefaultDBUnitMockStrutsTestCase with the handy Hibernate reader utility defined way (way!) up above.

public class ProjectListActionTest
  extends DefaultMerlinMockStrutsTestCase {

 public void testProjectListAction() throws Exception{
  this.setRequestPathInfo("/viewProjects");
  this.actionPerform();
  this.verifyForward("success");

  IProject[] projects = (IProject[])this.getRequest().
     getAttribute("projects");
  assertNotNull("object was null", projects);
 }

 public ProjectListActionTest(String arg0) {
  super(arg0);
 }

 protected String getDBUnitDataSetFileForSetUp() {
  return "dbunit-project-seed.xml";
 }
}

Now you have one excellent trippin’ test case, making it difficult for anyone to complain they can’t test this application in a repeatable manner!

Quality Labs just released PDbSeed, a Python framework for seeding a database. This is quite a hip tool during developer testing of applications which require a database. If you’ve ever heard of DbUnit, this is its Python cousin, man.

For example, here is a rough example of a PyUnit test case which refreshes the database on setup with a known dataset and then asserts the existence of a specific column value to verify the data was properly inserted.

import unittest
import pdbseed.core.dbseed as dbseed

class database_testcase(unittest.TestCase):

   def setUp(self):
     """
      setUp updates the database with the data found
      in the dataset
     """
     dbseed.RefreshOperation().execute(self._dataSet())

   def testData(self):
     """
      test case creates a new connection to db and
      asserts a specific item from the xml file exists
     """
     import MySQLdb
     conn= MySQLdb.connect( host = "localhost", \
        user = "words", passwd = "words", db = "words")

     query = 'select word.part_of_speech from word \
        where word.spelling = \'pugnacious\';'

     curs = conn.cursor()
     curs.execute(query)

     wlist = curs.fetchall()

     for word in wlist:
	  self.assertEqual('Adjective', word[0], 'Adj was not returned')

     conn.close()

   def _dataSet(self):
    """
      method returns a dbseed dataset type.
    """
    dbhost = 'localhost'
    dbname = 'words'
    dbuser = 'words'
    dbpassword = 'words'
    datasetFilename = 'C:/dev/prdbtesting/conf/words-seed.xml'
    metaDataFilename = 'C:/dev/prdbtesting/conf/metadata.xml'

    context = dbseed.ContextFactory.createContext('mysql', metaDataFilename, \
       dbhost, dbname, dbuser, dbpassword)

    return dbseed.FlatXMLFileRecordProducer(context, datasetFilename)

if __name__ == "__main__":
    unittest.main()

This example reads two files. One is metadata file which describes a database schema and the other is a dataset which contains values to be inserted into the database.

For more information, see the PDbSeed documentation; additionally, the library bundles some example code. Dig it?

The latest version of the Groovy plug-in for Eclipse is broken in CVS. It doesn’t compile and once it does, you can’t make GroovyTestCases or normal JUnit tests either.

If you want to be groovy with Groovy in Eclipse, you’ll need to follow the instructions on the Groovy site and do the following additional trippn’ tasks:

1. In the class org.codehaus.groovy.eclipse.editor.GroovyConfiguration delete the org.codehaus.groovy.eclipse.editor.GroovySourceViewerConfiguration.Hover import.
2. In the class org.codehaus.groovy.eclipse.model.GroovyProject fix the method isTestCaseClass to use the getSuperClassNode() method instead of the getSuperClass() one, like this:

   private boolean isTestCaseClass(ClassNode classNode) {
     ClassNode parent = classNode.getSuperClassNode();
     while (parent != null) {
      if (parent.getNameWithoutPackage().equals("TestCase")) {
       return true;
      }
      parent = parent.getSuperClassNode();
     }
     return false;
   }
   

3. Edit the plugin.xml file to include <import plugin="org.junit"/> in the <requires> section.

Export the plug-in and you should be good to go disco dancing (or at least Groovy programmin’).

Ever run a build that lasted 4 hours? Yeah, probably once, right? After that, you figured out how to merely compile your own hip stuff so you could actually get some work done and not have to wait around for the entire build saga to complete.

Then everyone else figured out how to do the same fab thing. Unfortunately, one day before the big release (and before you installed a CI system), someone actually ran the complete build and lo and behold, tests started failing (2 hours into the build, however).

Unless a build compiles millions (and millions) of files, the culprit of a prolonged build is usually the testing step (which could be a series of steps!). This aggregate time to run a series of tests has the additional tendency to become longer when there are extensive setup steps too, such as configuring a database, or deploying a .war file, to name a few.

For any nontrivial software project that wishes to keep build times bearable, it becomes paramount then to create an effective strategy for the categorization of tests. By segregating tests into categories and running the associated categories at prescribed intervals, build times can remain manageable for developers and CI systems, alike.

A true unit test should run to completion (successfully) in less than a few seconds. If a unit test takes longer, take a close look at it- it is either broken or really a component level test. Because unit tests run so quickly, they should be run anytime a build is run.

The whole groovy mantra of “code a little, test a little, code a little…” is predicated on the notion of rapid testing. If unit testing Bogarts enough time that someone can focus on something else, it’s taking too long! It will become a burden and soon forgotten.

In a CI environment, builds are run anytime someone changes a source repository; therefore, unit tests should be run anytime someone checks in code. There is no configuration cost and the resource consumption cost to run them is negligible.

Component tests, which usually have multiple dependencies, take a bit longer to run. As such, they should be run at regular intervals, but not necessarily every time a build is run, man. These tests have a cost to them- dependencies have to be put in place and configured; moreover, these tests alone may only take a few seconds, however, in the aggregate, this time adds up.

For example, the trippn’ component test below takes, on average, 4 seconds to run.

package test.org.aglover.words.dao.spring;

import java.util.Set;
import junit.framework.TestCase;
import org.aglover.words.bizobj.IWord;
import org.aglover.words.dao.WordDAO;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import com.vanward.resource.unittest.dbunit.hibernate.DefDBUnitTestCase;

public class SpringWordDAOImplTest extends DefDBUnitTestCase {

  public void testFindVerifyDefinitionsSize() throws Exception{
    ApplicationContext context =
        new ClassPathXmlApplicationContext("spring-config.xml");

    WordDAO dao = (WordDAO) context.getBean("wordDAO");
    IWord wrd = dao.findWord("pugnacious");
    Set defs = wrd.getDefinitions();
    TestCase.assertEquals("size should be one", 1, defs.size());
  }

  public SpringWordDAOImplTest(String arg0) {
    super(arg0);
  }

  protected String getDBUnitDataSetFileForSetUp() {
    return "words-seed.xml";
  }
}

This test does a number of things which add to the total test time and configuration complexity. First, this test seeds a database via DbUnit. In this case, DbUnit does an insert of the data found in the xml file words-seed.xml, which also implies that XML parsing is being done. This also assumes the test case can find the file easily.

Next, this test case configures Spring, which also configures Hibernate. Finally, a test is run and a word is retrieved from the database. Any wonder why this takes 4 seconds? Keep in mind, this is only one test case. Each additional test case in this class may not add another 4 seconds, however, it’ll probably add about 2 seconds. Do this about 10 more times and you have a minute. Get the picture?

While these types of tests shouldn’t be executed every time a build is run, it’s a good bet to run them before committing code into a repository. In a CI environment, it’s probably a good idea to run these at least a few times a day. Running them every time someone checks in code could cause issues; however, this takes good judgment. Some projects can get away with running component level tests in a CI environment anytime someone checks in code.

System and functional tests, which require a fully installed system, take the longest to run. Additionally, the complexity in configuring a fully functional system occasionally prohibits the full automation of these tests.

Ideally, developers can run these tests locally, when needed. In a CI environment, nightly (if they can be pulled off in an automated fashion) is a good bet with these tests. Running these tests frequently in a CI environment could a recipe for disaster and probably overkill if other tests are run often. Sometimes at the end of release cycles though, these types of tests can be run on a more frequent interval.

The next time you blindly add a test case to your build, consider the long term implications of running all of your tests and then start optimizing your build to categorize your tests so you can run them at varying frequencies. Everyone will thank you, especially the disco dancers in the crowd!

The latest version of the Groovy plug-in for IntelliJ IDEA (GroovyJ 0.1.7) doesn’t update the classpath to include JUnit; therefore, if you attempt to create a hip GroovyTestCase, you’ll get a strange error: unable to resolve class GroovyTestCase. Simply add JUnit to your classpath and that error will go away. Dig it?

The long awaited next hip version of JUnit was released today. This week’s SD Times issue has two aptly timed trippin’ articles on the 4.0 release as well. Kent Beck is interviewed in New Thinking for JUnit 4 and both Kent and some disco dancer were interviewed in JUnit 4 Adds Annotations.

During the boogie of development with tight schedules and impending Disco Dances, it’s tempting to try and fit everything into a test case. This haphazardness tends to lead to an abundance of assert methods ending up in one test case.

For example, the code below attempts to verify the behavior of HierarchyBuilder’s buildHierarchy method as well as the behavior of the Hierarchy object in one test case.

public void testBuildHierarchy() throws Exception{
  Hierarchy hier = HierarchyBuilder.buildHierarchy(
      "test.com.vanward.adana.hierarchy.HierarchyBuilderTest");

  assertEquals("should be 2", 2,
      hier.getHierarchyClassNames().length);   

  assertEquals("should be junit.framework.TestCase",
      "junit.framework.TestCase",
      hier.getHierarchyClassNames()[0]);  

  assertEquals("should be junit.framework.Assert",
      "junit.framework.Assert",
       hier.getHierarchyClassNames()[1]);
}

Note how there are three assert methods, man. This is a valid JUnit test case; there is nothing prohibiting the inclusion of multiple asserts in a test case.

The problem, however, with this practice is that JUnit is built to be fast failing. If the first assert fails, the whole test case is abandoned from the point of failure. This means that the next two tripping asserts are not run during that test run.
Once a code fix is completed and the test is rerun, the second assert may fail, which causes the whole fix-rerun test case cycle to repeat. If when running the second try, the third assert fails, yet again, the process repeats. Notice an inefficient, uptight pattern here?

A more effective practice is to try and limit one assert to each test case. This way, rather than repeating a three step process as you would in the example above, there would be three failures in one test run.

For example, the code from above would be refactored into three separate test cases (and a fixture).

private Hierarchy hier;

protected void setUp() throws Exception {
    hier = HierarchyBuilder.buildHierarchy(
      "test.com.vanward.adana.hierarchy.HierarchyBuilderTest");
}

public final void testBuildHierarchyStrSize() throws Exception{
    TestCase.assertEquals("should be 2", 2,
       hier.getHierarchyClassNames().length);
}

public final void testBuildHierarchyStrNameAgain() throws Exception{
    TestCase.assertEquals("should be junit.framework.TestCase",
       "junit.framework.TestCase",
       hier.getHierarchyClassNames()[0]);
}

public final void testBuildHierarchyStrName() throws Exception{
    TestCase.assertEquals("should be junit.framework.Assert",
       "junit.framework.Assert",
       hier.getHierarchyClassNames()[1]);
}

With three separate test cases, in the first test run, three failures are reported. This way, you can limit yourself to one fix-rerun cycle. This practice, of course, leads to a proliferation of test cases; however, there is a benefit to that: the number of test cases grows at a smokin’ pace!

There are tangible practices, such as developer testing and continuous integration, that decrease the frequency of software defects, but the fact of the matter is that defects will occur. That’s copasetic though- mistakes happen and mistakes can be forgiven. Peace, man. Making the same mistake twice, though, is quite unforgivable.

In the past few years, the term Defect Driven Development has become hip; however, that term has always sounded rather establishment. Defects don’t drive development- preventing those nasty aberrations drives development! If anything, defects halt development, man- it’s the act of addressing them and then assuring they don’t come back that keeps the wheels moving. Here is a proven strategy for guaranteeing that once a defect is found, it doesn’t come back.

When a defect is discovered, find and isolate the offending code. If the project has a healthy amount of test cases, it’s probably a good bet that the defect has occurred in some portion of untested code (which could be an unconsidered path) - most likely in the interaction of components.

For example, below is a find method in a Hibernate DAO class, which attempts to retrieve a word from a database.

public IWord findWord(String word) throws FindException{
  Session sess = null;
  try{
    sess = WordDAOImpl.sessFactory.getHibernateSession();

    final Query qry = sess.getNamedQuery("word.finder.bySpelling");
    qry.setString("spelling", word);					

    final List lst = qry.list();
    final IWord wrd = (IWord)lst.get(0);
    sess.close();
    return wrd;
  }catch(Throwable thr){
    try{sess.close();}catch(Exception e){}
    throw new FindException("Exception while finding word: "
        + word + " "+ thr.getMessage(), thr);
  }
}

This class has been reasonably tested in a series of component level tests which utilize DbUnit. These tests verify the basic CRUD (create, read, update, and delete) operations. For example, a test for the find method is shown below.

public void testFindVerifyDefinition() throws Exception{
  final WordDAOImpl dao = new WordDAOImpl();
  final IWord wrd = dao.findWord("pugnacious");

  for(Iterator iter = wrd.getDefinitions().iterator(); iter.hasNext();){
    IDefinition def = (IDefinition)iter.next();
    TestCase.assertEquals("def should be Combative in nature; belligerent.",
        "Combative in nature; belligerent.", def.getDefinition());
  }
}  

During functional testing of the larger application (in this case, a dictionary), it is discovered that if the user attempts to search for a word that isn’t in the dictionary, the application heaves a nasty exception stack trace, which utterly confuses users.

After some crafty detective work, someone discovers that the findWord method in WordDAOImpl throws an unexpected IndexOutOfBoundsException (which is masked by a FindException) if no word is returned via the Hibernate API.

This aberrant behavior wasn’t accounted for! A defect has been discovered! All’s not lost though. Remember, we are forgiven for creating this defect, but only once. Love Power, man. We have an opportunity to fix this nefarious glitch, but if it breaks again our skills may (should?) be called in to question.

The first step in regaining our pride is to write a test case which exposes the defect. Read that again slowly. Your first reaction may be to go fix the offending code and move on to other more exciting things (boogie at the local Disco!); however, if you go that route, you loose an excellent chance to ensure the same bug never comes back again.

Start by writing a test case which triggers the same exact behavior that was reported in the defect summary. In our case, we need to cause the code to throw an IndexOutOfBoundsException.

public void testFindInvalidWord() throws Exception{
  final WordDAOImpl dao = new WordDAOImpl();
  try{
     final IWord wrd = dao.findWord("fetit");
     TestCase.fail("This should throw an exception");
  }catch(FindException ex){
     Throwable thr = ex.getOriginalException();
     TestCase.assertTrue("Should be instance of IndexOutOfBoundsException",
        ex.getOriginalException() instanceof IndexOutOfBoundsException);
  }
}

If we run the test above, it passes. Therefore, we’ve proven there is a defect. Now we can fix it.

This methodology, by the way, is slightly different than the prevailing “Defect Driven Development” mantra, which suggests writing a failing test case first and then keep running that test (while fixing the defect) until the test stops failing.

For example, the code below is a Defect Driven test case.

public void testFindInvalidWordException() {
  final WordDAOImpl dao = new WordDAOImpl();
  try{
     final IWord wrd = dao.findWord("fetit");
  }catch (FindException e){
     TestCase.fail("Didn't find word fetit");
  }
}

This test case, of course, fails when first run (assuming the defect is still present). This practice does work; however, it presents some opportunities for refinement.

The challenges with writing a test case that purposely fails at first are:

1. It is difficult to write a failing test in this scenario that uses an assert properly.
   • Because of this, asserts may not ever be added, even after the tests case doesn’t fail anymore. This means the test case isn’t necessarily passing- it is merely not failing.
2. At this point in the game, it is tricky to know how the fix will affect behavior, so in attempting to fail the test you end up guessing on what the fix may be. Dig it?
   • In the example, the assumption is made that the fix is that the code won’t throw an exception anymore. This is true, but only part of the whole story.
3. Once a fix has been made in the code under test, the failing test works; however, it doesn’t actually verify the change in behavior.
   • At this point, because the test case works, most people don’t go back to update it.

In our case, to fix the defect, we, in essence, need to break the test, which is opposite from the other theory. Tripped out yet?

Examining the code closely reveals we need to check for an empty list before attempting to grab the first element. We’re left with a design choice, however, at this point- should the code return null or an empty Word or throw an exception?

The decision is made to return null if the parameter value can not be retrieved via Hibernate from the database as shown below.

public IWord findWord(String word) throws FindException{
  Session sess = null;
  try{
     sess = WordDAOImpl.sessFactory.getHibernateSession();

     final Query qry = sess.getNamedQuery("word.finder.bySpelling");
     qry.setString("spelling", word);					

     final List lst = qry.list();
     IWord wrd = null;
     if(lst.size() > 0){
       wrd = (IWord)lst.get(0);
     }
     sess.close();
     return wrd;
  }catch(Throwable thr){
     try{sess.close();}catch(Exception e){}
     throw new FindException("Exception while finding word: "
         + word + " "+ thr.getMessage(), thr);
  }
}

With the code under test conceivably fixed, the test is run again and this time it fails. Now our next decision is what differentiates this methodology from the rest of the word- in fixing our test case, we will assert the new behavior. The Defect Driven example would work by now and the chances are, we’d leave the test case as so. But that test case doesn’t provide too much value now.

We need to assert that when an invalid word is passed into the findWord method, null is returned. Furthermore, we also need to assert than an Exception isn’t thrown. The updated test case is shown below.

public void testFindInvalidWord() throws Exception{
  final WordDAOImpl dao = new WordDAOImpl();
  try{
     final IWord wrd = dao.findWord("fetit");
     TestCase.assertNull("Should have received back a null object", wrd);
  }catch(FindException ex){
     TestCase.fail("This should not throw an exception");
  }
}

Now we’re done (and in the groove) and we’ve accomplished two things: first, the defect has been corrected. Disco, way to go, you are smokin’. Second, a regression test is now in place that truly asserts the correct behavior of the fix.

Defect Driven Development or Regression Prevention Development? They both drive you to:

1. Fix the defect
2. Prevent the defect from recurring

Regression Prevention Development, however, has the tendency to drive you to carry out a third step, which is asserting any new behavior triggered by the defect’s fix. Outta sight!

Quality Labs has released a Maven plug-in for FIT, which enables Maven builds to run FIT tests automatically. FIT is a testing platform that facilitates communication between those who write requirements and those smokin’ cats who turn them into code. The plug-in invokes FIT’s FolderRunner, which accepts a location for FIT tables and a location to write the resulting report. Far out!

Stay tuned for an article on FIT in the next tripped out installment of In Pursuit of Code Quality on IBM’s DeveloperWorks!