Making metrics work
Over on ontoinfo.com, an interesting article was published entitled “Making Software Quality Metrics Actually Work” in which the author states that “No one quality metric can provide [an] accurate and complete picture.” Bingo! They hit the nail on the head with that one, man.
A metric, in and of itself, is not very useful– for example, code coverage is simply a number that, in isolation, doesn’t mean anything. Highly covered code is still susceptible to defects! Coverage also is not an indication of test quality, by the way. Coverage is simply a measure of test execution– what’s important is how the coverage values are utilized and compared against other hip metrics or trends.
The obvious immediate value of coverage reports is not in what has been tested, but in what’s not tested. Monitoring this value, in concert with complexity, is a valuable strategy that enables teams to quickly ascertain areas of associated risk (i.e. areas of code with high complexity and low code coverage should be a focus point– either through additional developer testing or even coordination with QA efforts).
Maintenance metrics is another grouping of copasetic metrics that work well in concert. Coupling alone is not all that helpful– it’s when these metrics are combined with Abstractness or Instability or even code coverage that a window into risk can be viewed. And only then, when metrics are viewed in relation to other metrics or even when viewed over time can you actually begin to make metrics work. Dig it?
| Related odds and ends | ||
|---|---|---|
5 comments Friday 06 Oct 2006 | Code Metrics
5 Responses to “Making metrics work”
Andy, what you describe here is very true. That is why I started toying with the idea of a tool that gathers information from metrics tools (like xradar does).
The first stage groups javanss with JDepend; next will be coverage data (I’m thinking Cobertura) and probably will mix-in cvs/svn access for developer data.
Very cool! Another interesting tool for Java, which provides some interesting metrics, is PMD.
Thanks for the suggestion, PMD is also on the list with other code analizers like FindBugs, Lint4J and Checkstyle. What I have in mind is a dashboard that groups all the data in one place and will let you discover relations on bugs, code coverage and metrics.
By the way, Jason Bennett just wrote an NPath Complexity rule for PMD, so that’ll make its way into the next release. Good times…
Interesting metric, Tom. It’ll be interesting to see how NPath’s numbers related to PMD’s Cyclomatic Complexity ones. Based upon what I’ve been able to find related to NPath it seems like a more precise CCN.