Many web applications work against databases, man. Databases, however, present quite a large dependency for testing, leaving one with two choices- either mock out as much as possible and avoid the database all together for as long as possible or suck it up and utilize the database. The latter choice presents a new series of challenges- how does one control the database during testing? Even better, how does one make those tests repeatable?

By far the easiest way to make your testing cake and eat it is to use a database seeding framework like any of the xDbUnits. These hip frameworks abstract a database’s dataset into XML files and then offer the developer fine grained control as to how this data is seeded into a database during testing.

For example, the following snippet is from a DbUnit XML seed file.

 <word WORD_ID="1" SPELLING="pugnacious" PART_OF_SPEECH="Adjective"/>
 <definition DEFINITION_ID="10"
   DEFINITION="Combative in nature; belligerent."
   WORD_ID="1"
   EXAMPLE_SENTENCE="The pugnacious youth had no friends left to pick on."/>
 <synonym SYNONYM_ID="20" WORD_ID="1" SPELLING="belligerent"/>
 <synonym SYNONYM_ID="21" WORD_ID="1" SPELLING="aggressive"/>  

Via DbUnit’s DatabaseTestCase, the data in the XML file is manipulated via operations such as insert, update, and delete. The specific database is configured by implementing the abstract getConnection method and the XML file is located via the getDataSet method. You can dig it below.

package test.org.aglover.words.dao;

import java.io.File;
import java.sql.Connection;
import java.sql.DriverManager;
import java.util.Iterator;
import junit.framework.TestCase;
import org.aglover.words.bizobj.IDefinition;
import org.aglover.words.bizobj.IWord;
import org.aglover.words.dao.exception.FindException;
import org.aglover.words.dao.impl.WordDAOImpl;
import org.dbunit.DatabaseTestCase;
import org.dbunit.database.DatabaseConnection;
import org.dbunit.database.IDatabaseConnection;
import org.dbunit.dataset.IDataSet;
import org.dbunit.dataset.xml.FlatXmlDataSet;

public class DefaultWordDAOImplTest extends DatabaseTestCase {
    /**
     * @see org.dbunit.DatabaseTestCase#getDataSet()
     */
    protected IDataSet getDataSet() throws Exception {
        return new FlatXmlDataSet(
          new File("test/conf/words-seed.xml"));
    }
    /**
     * @see org.dbunit.DatabaseTestCase#getConnection()
     */
    protected IDatabaseConnection getConnection() throws Exception {
        final Class driverClass =
              Class.forName("org.gjt.mm.mysql.Driver");

        final Connection jdbcConnection =
       	 DriverManager.getConnection(
              "jdbc:mysql://localhost/words",
       	      "words", "words");
         return new DatabaseConnection(jdbcConnection);
    }

    public void testFindVerifyDefinition() throws Exception{
       final WordDAOImpl dao = new WordDAOImpl();
       final IWord wrd = dao.findWord("pugnacious");

       for(Iterator iter = wrd.getDefinitions().iterator(); iter.hasNext();){
         IDefinition def = (IDefinition)iter.next();
         assertEquals("Combative in nature; belligerent.",
              "Combative in nature; belligerent.",
               def.getDefinition());
       }
    }  

    public DefaultWordDAOImplTest(String arg0) {
        super(arg0);
    }
}

Note though, that this trippin’ class makes the assumption that the database is located on the same machine that the test is run. This maybe a safe assumption on the desktop, but obviously this configuration can present a challenge in CI environments.

One solution is to pull out the hard coded connection strings and place them into properties files. There is however, a more effective mechanism. If DbUnit is utilized to seed a database, one can infer that the application itself then uses a database. If this is the case, it is a common best practice to avoid hard coding connection information within a code base; therefore, why not configure DbUnit to read the same file that the application under test reads? Far out!

For example, in Hibernate applications, database connection information is usually defined in the hibernate.cfg.xml file. One can easily write a utility class which parses this file and obtains the proper connection information. Even better, one can rely on Hibernate to provide the desired information as shown below.

//imports left out

public class DBUnitHibernateConfigurator {
 static Configuration configuration = null;

 private DBUnitHibernateConfigurator() {
  super();
 }

 private static Configuration getConfiguration()
	throws HibernateException {
  if (configuration == null) {
   configuration = new Configuration().configure();
  }
  return configuration;
}

public static IDataSet getDataSet(final String fileName)
    throws ResourceNotFoundException,
     DBUnitHibernateConfigurationException {
 try{
  return DBUnitConfigurator.getDataSet(fileName);
 }catch(DBUnitConfigurationException e2){
   throw new DBUnitHibernateConfigurationException(
      "DBUnitConfigurationException in getDataSet", e2);
 }
}

 private static String getProperty(final String name)
    throws HibernateException {
  return getConfiguration().getProperty(name);
 }

public static Properties getHibernateProperties()
    throws ResourceNotFoundException,
      DBUnitHibernateConfigurationException{
 try{
  final Properties hProp = new Properties();
  hProp.put("hibernate.connection.driver_class",
    DBUnitHibernateConfigurator.getProperty(
      "hibernate.connection.driver_class"));
  hProp.put("hibernate.connection.url",
    DBUnitHibernateConfigurator.getProperty(
      "hibernate.connection.url"));
  hProp.put("hibernate.connection.username",
    DBUnitHibernateConfigurator.getProperty(
      "hibernate.connection.username"));
  hProp.put("hibernate.connection.password",
    DBUnitHibernateConfigurator.getProperty(
      "hibernate.connection.password"));
  return hProp;
 }catch(HibernateException e){
   throw new DBUnitHibernateConfigurationException(
     "HibernateException in getHibernatePropertiesFile", e);
 }
}

 public static IDatabaseConnection getDBUnitConnection()
    throws DBUnitHibernateConfigurationException{
  try{
   final Properties props =
     DBUnitHibernateConfigurator.getHibernateProperties();
   return DBUnitConfigurator.getDBUnitConnection(props);
  }catch(DBUnitConfigurationException e1){
   throw new DBUnitHibernateConfigurationException(
     "DBUnitConfigurationException in getDBUnitConnection", e1);
  }catch (ResourceNotFoundException e2) {
   throw new DBUnitHibernateConfigurationException(
     "ResourceNotFoundException in getDBUnitConnection", e2);
  }
 }
}

Note how the above class puts the Hibernate connection info in a Properties object, which is then converted into DbUnit’s IDatabaseConnection type in a DBUnitConfigurator class. The DbUnit connection type is then returned via the getDBUnitConnection method. DbUnit’s IDataSet type, which represents those XML files containing all the data is returned via the getDataSet method. This copasetic method frees developers from having to provide a path to a file- something that can be tricky in different environments.

Next, a custom abstract test case class can be created which requests that implementers feed the desired dataset information for a particular test case.

//left out imports...

public abstract class DefaultDBUnitHibernateTestCase extends DatabaseTestCase {
 public DefaultDBUnitHibernateTestCase(String arg0) {
  super(arg0);
 }	

  protected void setUp() throws Exception {
   super.setUp();
   DefaultHibernateSessionFactory.
      closeSessionAndEvictCache();
   DefaultHibernateSessionFactory.
      getInstance().getHibernateSession();
  }

  protected void tearDown() throws Exception {
   DefaultHibernateSessionFactory.
      closeSessionAndEvictCache();
   super.tearDown();
 }

  protected IDatabaseConnection getConnection() throws Exception {
   return DBUnitHibernateConfigurator.
      getDBUnitConnection();
  }

  protected IDataSet getDataSet() throws Exception {
   final String fileName = this.getDBUnitDataSetFileForSetUp();
   DatabaseTestCase.assertNotNull("data set file was null", fileName);
   return DBUnitHibernateConfigurator.getDataSet(fileName);
  }

  protected abstract String getDBUnitDataSetFileForSetUp();
}

An example resulting test case which implements DefaultDBUnitHibernateTestCase is shown below.

//imports left out 

public class WordDAOImplTest extends DefaultDBUnitHibernateTestCase {

 public void testUpdateWordSpelling() throws Exception{
  WordDAOImpl dao = new WordDAOImpl();
  IWord wrd = dao.findWord("pugnacious");

  wrd.setSpelling("pugnacious-ness");
  dao.updateWord(wrd);

  IWord wrd2 = dao.findWord("pugnacious-ness");
  assertEquals("should be id of 1", 1, wrd2.getId());
 }

 public void testFindVerifyDefinitionsSize() throws Exception{
  WordDAOImpl dao = new WordDAOImpl();
  IWord wrd = dao.findWord("pugnacious");

  Set defs = wrd.getDefinitions();
  assertEquals("size should be one", 1, defs.size());
 }

 protected String getDBUnitDataSetFileForSetUp() {
  return "words-seed.xml";
 }

 public WordDAOImplTest(String arg0) {
  super(arg0);
 }
}

DbUnit offers an API (as seen above), which can be utilized effectively via composition- creating enormous opportunities for powerful combination frameworks too. With this added flexibility, testing various architectures at different layers in the Age of Aquarius becomes quite easy.

For example, developer testing of Struts applications can be challenging. A common tactic is to utilize a framework like HttpUnit, which simulates http requests; however, this can be tedious work and doesn’t offer the precision one may desire in a Struts architecture which heavily utilizes Action classes and a configuration for mapping requests.

The StrutsTestCase project was created to address this issue- with this framework one can easily isolate and test Struts’ Action classes. This project, however, requires a developer extend a base class which handles mocking of a servlet container. If a Struts application requires the use of a database you may be left in a quandary.

Via DbUnit’s API, a combination framework can be created which utilizes the seeding capabilities of DbUnit with the mocking capabilities of the StrutsTestCase project.

package com.vanward.resource.unittest.dbunit.struts;

import java.util.Properties;
import org.dbunit.DatabaseTestCase;
import org.dbunit.database.IDatabaseConnection;
import org.dbunit.dataset.IDataSet;
import org.dbunit.operation.DatabaseOperation;
import servletunit.struts.MockStrutsTestCase;
import com.vanward.resource.unittest.dbunit.\
   util.DBUnitConfigurator;

public abstract class DefaultDBUnitMockStrutsTestCase
   extends MockStrutsTestCase {

 public DefaultDBUnitMockStrutsTestCase(String testName) {
  super(testName);
 }

 public void setUp() throws Exception {
  super.setUp();
  this.executeOperation(this.getSetUpOperation());
 }

 public void tearDown() throws Exception{
  super.tearDown();
  this.executeOperation(this.getTearDownOperation());
 }

 private void executeOperation(DatabaseOperation operation)
    throws Exception{
  if (operation != DatabaseOperation.NONE){
   final IDatabaseConnection connection =
     this.getConnection();
  try{
   operation.execute(connection, this.getDataSet());
  }finally{
   closeConnection(connection);
  }
 }
}

 protected void closeConnection(IDatabaseConnection connection)
   throws Exception{
  connection.close();
 }

 protected abstract Properties getConnectionProperties();

 protected abstract String getDBUnitDataSetFileForSetUp();

 protected IDatabaseConnection getConnection() throws Exception {
  final Properties dbPrps = this.getConnectionProperties();
  DatabaseTestCase.
    assertNotNull("database properties were null", dbPrps);
  return DBUnitConfigurator.getDBUnitConnection(dbPrps);
 }

 protected DatabaseOperation getSetUpOperation() throws Exception {
  return DatabaseOperation.CLEAN_INSERT;
 }

 protected DatabaseOperation getTearDownOperation() throws Exception {
  return DatabaseOperation.NONE;
 }

 protected IDataSet getDataSet() throws Exception {
  final String fileName = this.getDBUnitDataSetFileForSetUp();
  DatabaseTestCase.assertNotNull("data set file was null", fileName);
  return DBUnitConfigurator.getDataSet(fileName);
 }
}

One again, you may be left with the option of hardcoding connection information or reusing existing files for this purpose! Testing a Struts application that uses Hibernate? Not a problem- just combine the new DefaultDBUnitMockStrutsTestCase with a handy dandy utility for reading Hibernate files.

For example, below is a class which implements a DefaultMerlinMockStrutsTestCase class, which combines the DbUnit ability of DefaultDBUnitMockStrutsTestCase with the handy Hibernate reader utility defined way (way!) up above.

public class ProjectListActionTest
  extends DefaultMerlinMockStrutsTestCase {

 public void testProjectListAction() throws Exception{
  this.setRequestPathInfo("/viewProjects");
  this.actionPerform();
  this.verifyForward("success");

  IProject[] projects = (IProject[])this.getRequest().
     getAttribute("projects");
  assertNotNull("object was null", projects);
 }

 public ProjectListActionTest(String arg0) {
  super(arg0);
 }

 protected String getDBUnitDataSetFileForSetUp() {
  return "dbunit-project-seed.xml";
 }
}

Now you have one excellent trippin’ test case, making it difficult for anyone to complain they can’t test this application in a repeatable manner!

The long awaited next hip version of JUnit was released today. This week’s SD Times issue has two aptly timed trippin’ articles on the 4.0 release as well. Kent Beck is interviewed in New Thinking for JUnit 4 and both Kent and some disco dancer were interviewed in JUnit 4 Adds Annotations.

During the boogie of development with tight schedules and impending Disco Dances, it’s tempting to try and fit everything into a test case. This haphazardness tends to lead to an abundance of assert methods ending up in one test case.

For example, the code below attempts to verify the behavior of HierarchyBuilder’s buildHierarchy method as well as the behavior of the Hierarchy object in one test case.

public void testBuildHierarchy() throws Exception{
  Hierarchy hier = HierarchyBuilder.buildHierarchy(
      "test.com.vanward.adana.hierarchy.HierarchyBuilderTest");

  assertEquals("should be 2", 2,
      hier.getHierarchyClassNames().length);   

  assertEquals("should be junit.framework.TestCase",
      "junit.framework.TestCase",
      hier.getHierarchyClassNames()[0]);  

  assertEquals("should be junit.framework.Assert",
      "junit.framework.Assert",
       hier.getHierarchyClassNames()[1]);
}

Note how there are three assert methods, man. This is a valid JUnit test case; there is nothing prohibiting the inclusion of multiple asserts in a test case.

The problem, however, with this practice is that JUnit is built to be fast failing. If the first assert fails, the whole test case is abandoned from the point of failure. This means that the next two tripping asserts are not run during that test run.
Once a code fix is completed and the test is rerun, the second assert may fail, which causes the whole fix-rerun test case cycle to repeat. If when running the second try, the third assert fails, yet again, the process repeats. Notice an inefficient, uptight pattern here?

A more effective practice is to try and limit one assert to each test case. This way, rather than repeating a three step process as you would in the example above, there would be three failures in one test run.

For example, the code from above would be refactored into three separate test cases (and a fixture).

private Hierarchy hier;

protected void setUp() throws Exception {
    hier = HierarchyBuilder.buildHierarchy(
      "test.com.vanward.adana.hierarchy.HierarchyBuilderTest");
}

public final void testBuildHierarchyStrSize() throws Exception{
    TestCase.assertEquals("should be 2", 2,
       hier.getHierarchyClassNames().length);
}

public final void testBuildHierarchyStrNameAgain() throws Exception{
    TestCase.assertEquals("should be junit.framework.TestCase",
       "junit.framework.TestCase",
       hier.getHierarchyClassNames()[0]);
}

public final void testBuildHierarchyStrName() throws Exception{
    TestCase.assertEquals("should be junit.framework.Assert",
       "junit.framework.Assert",
       hier.getHierarchyClassNames()[1]);
}

With three separate test cases, in the first test run, three failures are reported. This way, you can limit yourself to one fix-rerun cycle. This practice, of course, leads to a proliferation of test cases; however, there is a benefit to that: the number of test cases grows at a smokin’ pace!

There are tangible practices, such as developer testing and continuous integration, that decrease the frequency of software defects, but the fact of the matter is that defects will occur. That’s copasetic though- mistakes happen and mistakes can be forgiven. Peace, man. Making the same mistake twice, though, is quite unforgivable.

In the past few years, the term Defect Driven Development has become hip; however, that term has always sounded rather establishment. Defects don’t drive development- preventing those nasty aberrations drives development! If anything, defects halt development, man- it’s the act of addressing them and then assuring they don’t come back that keeps the wheels moving. Here is a proven strategy for guaranteeing that once a defect is found, it doesn’t come back.

When a defect is discovered, find and isolate the offending code. If the project has a healthy amount of test cases, it’s probably a good bet that the defect has occurred in some portion of untested code (which could be an unconsidered path) – most likely in the interaction of components.

For example, below is a find method in a Hibernate DAO class, which attempts to retrieve a word from a database.

public IWord findWord(String word) throws FindException{
  Session sess = null;
  try{
    sess = WordDAOImpl.sessFactory.getHibernateSession();

    final Query qry = sess.getNamedQuery("word.finder.bySpelling");
    qry.setString("spelling", word);					

    final List lst = qry.list();
    final IWord wrd = (IWord)lst.get(0);
    sess.close();
    return wrd;
  }catch(Throwable thr){
    try{sess.close();}catch(Exception e){}
    throw new FindException("Exception while finding word: "
        + word + " "+ thr.getMessage(), thr);
  }
}

This class has been reasonably tested in a series of component level tests which utilize DbUnit. These tests verify the basic CRUD (create, read, update, and delete) operations. For example, a test for the find method is shown below.

public void testFindVerifyDefinition() throws Exception{
  final WordDAOImpl dao = new WordDAOImpl();
  final IWord wrd = dao.findWord("pugnacious");

  for(Iterator iter = wrd.getDefinitions().iterator(); iter.hasNext();){
    IDefinition def = (IDefinition)iter.next();
    TestCase.assertEquals("def should be Combative in nature; belligerent.",
        "Combative in nature; belligerent.", def.getDefinition());
  }
}  

During functional testing of the larger application (in this case, a dictionary), it is discovered that if the user attempts to search for a word that isn’t in the dictionary, the application heaves a nasty exception stack trace, which utterly confuses users.

After some crafty detective work, someone discovers that the findWord method in WordDAOImpl throws an unexpected IndexOutOfBoundsException (which is masked by a FindException) if no word is returned via the Hibernate API.

This aberrant behavior wasn’t accounted for! A defect has been discovered! All’s not lost though. Remember, we are forgiven for creating this defect, but only once. Love Power, man. We have an opportunity to fix this nefarious glitch, but if it breaks again our skills may (should?) be called in to question.

The first step in regaining our pride is to write a test case which exposes the defect. Read that again slowly. Your first reaction may be to go fix the offending code and move on to other more exciting things (boogie at the local Disco!); however, if you go that route, you loose an excellent chance to ensure the same bug never comes back again.

Start by writing a test case which triggers the same exact behavior that was reported in the defect summary. In our case, we need to cause the code to throw an IndexOutOfBoundsException.

public void testFindInvalidWord() throws Exception{
  final WordDAOImpl dao = new WordDAOImpl();
  try{
     final IWord wrd = dao.findWord("fetit");
     TestCase.fail("This should throw an exception");
  }catch(FindException ex){
     Throwable thr = ex.getOriginalException();
     TestCase.assertTrue("Should be instance of IndexOutOfBoundsException",
        ex.getOriginalException() instanceof IndexOutOfBoundsException);
  }
}

If we run the test above, it passes. Therefore, we’ve proven there is a defect. Now we can fix it.

This methodology, by the way, is slightly different than the prevailing “Defect Driven Development” mantra, which suggests writing a failing test case first and then keep running that test (while fixing the defect) until the test stops failing.

For example, the code below is a Defect Driven test case.

public void testFindInvalidWordException() {
  final WordDAOImpl dao = new WordDAOImpl();
  try{
     final IWord wrd = dao.findWord("fetit");
  }catch (FindException e){
     TestCase.fail("Didn't find word fetit");
  }
}

This test case, of course, fails when first run (assuming the defect is still present). This practice does work; however, it presents some opportunities for refinement.

The challenges with writing a test case that purposely fails at first are:

1. It is difficult to write a failing test in this scenario that uses an assert properly.
   • Because of this, asserts may not ever be added, even after the tests case doesn’t fail anymore. This means the test case isn’t necessarily passing- it is merely not failing.
2. At this point in the game, it is tricky to know how the fix will affect behavior, so in attempting to fail the test you end up guessing on what the fix may be. Dig it?
   • In the example, the assumption is made that the fix is that the code won’t throw an exception anymore. This is true, but only part of the whole story.
3. Once a fix has been made in the code under test, the failing test works; however, it doesn’t actually verify the change in behavior.
   • At this point, because the test case works, most people don’t go back to update it.

In our case, to fix the defect, we, in essence, need to break the test, which is opposite from the other theory. Tripped out yet?

Examining the code closely reveals we need to check for an empty list before attempting to grab the first element. We’re left with a design choice, however, at this point- should the code return null or an empty Word or throw an exception?

The decision is made to return null if the parameter value can not be retrieved via Hibernate from the database as shown below.

public IWord findWord(String word) throws FindException{
  Session sess = null;
  try{
     sess = WordDAOImpl.sessFactory.getHibernateSession();

     final Query qry = sess.getNamedQuery("word.finder.bySpelling");
     qry.setString("spelling", word);					

     final List lst = qry.list();
     IWord wrd = null;
     if(lst.size() > 0){
       wrd = (IWord)lst.get(0);
     }
     sess.close();
     return wrd;
  }catch(Throwable thr){
     try{sess.close();}catch(Exception e){}
     throw new FindException("Exception while finding word: "
         + word + " "+ thr.getMessage(), thr);
  }
}

With the code under test conceivably fixed, the test is run again and this time it fails. Now our next decision is what differentiates this methodology from the rest of the word- in fixing our test case, we will assert the new behavior. The Defect Driven example would work by now and the chances are, we’d leave the test case as so. But that test case doesn’t provide too much value now.

We need to assert that when an invalid word is passed into the findWord method, null is returned. Furthermore, we also need to assert than an Exception isn’t thrown. The updated test case is shown below.

public void testFindInvalidWord() throws Exception{
  final WordDAOImpl dao = new WordDAOImpl();
  try{
     final IWord wrd = dao.findWord("fetit");
     TestCase.assertNull("Should have received back a null object", wrd);
  }catch(FindException ex){
     TestCase.fail("This should not throw an exception");
  }
}

Now we’re done (and in the groove) and we’ve accomplished two things: first, the defect has been corrected. Disco, way to go, you are smokin’. Second, a regression test is now in place that truly asserts the correct behavior of the fix.

Defect Driven Development or Regression Prevention Development? They both drive you to:

1. Fix the defect
2. Prevent the defect from recurring

Regression Prevention Development, however, has the tendency to drive you to carry out a third step, which is asserting any new behavior triggered by the defect’s fix. Outta sight!