The Disco Blog Welcome Guest | Login or Register

As I’ve written about before, code coverage numbers can be misleading. 100% line coverage and 100% branch coverage doesn’t necessarily mean your code is defect free– all it means is that that code was touched. In fact, code coverage values are far more effective at telling you what’s not covered by a test, man.

There are actually quite a few hip different ways that coverage values can be misleading. One particular coding construct can easily mislead the unsuspecting eye: the short-circuit operator. These operators, such as the short-circuit AND (&&) and OR (||) are quite handy in conditionals. For example, here’s a fictional code snippet with a copasetic OR short-circuit operator in the first conditional.

public void branchIt(int value){
 if((value > 100) || (HiddenObject.doWork() == 0)){
  this.doIt();
 }else{
  this.doItAgain();
 }
}  

In the snippet above, both the doIt and the doItAgain methods aren’t particularly important– what’s key here is the second part of the if conditional. Let’s imagine HiddenObject is a 3rd party API call, an object in another package, etc– the point being, you don’t necessarily know how doWork does its work, you just know that because it’s its bag that if it returns 0 it is a valid condition.

It just so turns out that the doWork method on the HiddenObject method isn’t perfect– it can throw an exception. I’ll force one, but the scenario demonstrates a trippin’ point.

public static int doWork(){
 throw new RuntimeException("surprise!");
}

Thus far we know that if the doWork method is executed an exception will be thrown. Imagine I don’t know that though. Let me write a quick test (via JUnit) to make sure things are working.

public final void testBranchIt() {
 AnotherBranchCoverage clzzUnderTst = new AnotherBranchCoverage();
 clzzUnderTst.branchIt(101);
}

When I run this test, things work fine. I can even check out the coverage values as reported by Cobertura.

Not bad– I’ve got reasonable line coverage here and 100% branch coverage. It turns out the 100% value for branch coverage is a slight defect within Cobertura, but regardless, I’ve got copasetic coverage, don’t I? Check it out, man, the if statement was touched! Because of the short-circuit OR, I triggered a true condition via the 101 value; accordingly, the second clause was short-circuited. The 75% line coverage makes sense too– I failed to execute the else block, hence the other method within the class wasn’t touched and the line coverage value was accordingly deducted.

As you can see, coverage reports are hip in ascertaining what’s not tested (which, in this case, would practically force me to execute the 2nd condition in the if) but don’t depend too heavily on them telling you what’s tested. Otherwise you could end up short-circuiting yourself into a false sense of security…or is that coverage?

Trackback this Post | Feed on comments to this Post