The Disco Blog

As new tests are added to a code base, the build time will invariably increase. As I have written about before, a process of copasetic test categorization can be employed to help manage build times and thus, test frequencies. As it turns out, test categorization is easy to implement in NUnit using the aptly named Category attribute.

While a particular nomenclature isn’t terribly important, I do believe it essential to standardize on a common vocabulary and understand the neat-o concepts related to each. Briefly, I recommend the following three hip categories:

• Unit tests verify the behavior of small elements in a software system, which are most often a single class. A unit test should run to completion (successfully) in a fraction of a second and should be run anytime a build is run.
• Component tests test portions of a system and may require a fully installed system or a more limited set of external dependencies, such as databases or file systems to name a few. Because these smokin’ tests usually have multiple dependencies, they take a bit longer to run and should be run at regular intervals.
• System tests exercise a complete software system. Because these tests exercise an entire system, they are often created towards the latter cycles of development; furthermore, these tests have the tendency to have lengthy run times, in addition to prolonged set up times.

NUnit’s tripping category attribute can be applied to both classes (i.e a TestFixture) and test methods (i.e. a Test). For example, the following C# test, which verifies an object’s Equals implementation, has been categorized as belonging to the unit group.

[Test]
[Category("unit")]
public void VerifyEquals()
{
  Word wrd1 = new Word();
  wrd1.Spelling = "cerebration";
  wrd1.PartOfSpeech = "noun";

  Word wrd2 = new Word();
  wrd2.PartOfSpeech = "noun";
  wrd2.Spelling = "cerebration";

  Assert.IsTrue(wrd1.Equals(wrd2), "words were not equal!");
}

Additionally, an entire class can be associated with a group by declaring the Category attribute at the TestFixture level as demonstrated below:

[TestFixture]
[Category("component")]
public class WordsTest
{
  //.....
}

Once tests have been grouped accordingly, various NUnit runners can be configured to run a selected group or set of groups. For example, configuring a desired group to execute via NAnt is as easy as creating four targets (each relating to a particular group, plus one for all three).

Below, the logic for running all component tests is captured in the <categories> sub-element of NUnit’s <test> element.

<target name="component-test" depends="build">
 <nunit2 failonerror="false">
  <formatter type="Xml" usefile="true" extension=".xml"
       outputdir="${build.dir}/results/component/" />
  <test assemblyname="${build.dir}\bin\${build.config}\words.dll"
        appconfig="mydefaulttest.config">

    <categories>
      <include name="component" />
    </categories>

  </test>
 </nunit2>
</target>

As part of integrating this categorization within a CI system, the frequencies from above are mapped to a particular target- for instance, the unit-test target (which runs unit tests) can be run anytime the CM system changes (i.e. someone checked something in).

Test categorization within NUnit couldn’t be simpler and by segregating tests into logical groups defined by how long they take to run, build times can be kept manageable as a code base’s tests grow. Dig it?