Groovy’s the elixir for report overload syndrome
Not long ago, I posted a poll regarding code metrics in which the majority of votes settled on two, not too surprising, copasetic points:
- Some people are not sure what the data means
- Others aren’t sure which tools to use to obtain the data
Issue #1 spawned a posting a few weeks back regarding the meaning of code metrics; however, issue #2 got me thinking– there are quite a few different tools out there that gather diverse metrics, yet there are few opportunities to effectively view them. For instance, for Java projects, I often find myself recommending people use JavaNCSS, Cobertura, and PMD to name a few hip tools. In fact, in all, one can categorize various tools’ outputs into seven metric types:
- Test results
- Code coverage
-
- Tools like Cobertura report how much of the code is actually exercised by tests.
- Complexity
-
- JavaNCSS and PMD nicely flag methods with high Cyclomatic complexities.
- Code size
-
- JavaNCSS and PMD also report code sizes data. JavaNCSS indicates the number of classes, methods, lines of code and PMD can report classes and methods that are considered too big. This is closely related to complexity too.
- Coupling
-
- JDepend reports how many package dependencies are within a code base.
- Duplication
- Coding standards
-
- FindBugs, PMD, and CheckStyle report violations to predefined rules that may indicate possible defects or maintenance challenges.
This list, by the way, is composed of Paul Duvall’s Big Five code analysis areas– because it’s my bag, I’ve added test results and code size.
But looking at the list above reveals over six different reports, each with different formats and variations on how data is visually displayed. This cornucopia of data often leads to report overload syndrome, in which, because of information overload, the data has the tendency to become ignored (not much unlike the lack of an Oscar for the ever so hip Saturday Night Fever).
As I was sitting on a plane recently I found myself looking for an easy way to disseminate the valuable data from the categories above in an effective manner. I ended up with a design of a small table that displays summary data from the various tools and which provides links to the actual tool’s report for a more in depth analysis, should one feel it warranted.
As you can see from the image above, the table summarizes the output of seven tools used in a typical build:
- Test results from JUnit or TestNG
- JavaNCSS’s count of classes and lines of code as well as the maximum cyclomatic complexity found in an individual method
- Cobertura’s line and branch coverage
- The count of FindBugs violations as well as PMD’s count
- JDepend’s maximum reported Afferent and Efferent coupling
- The amount of code which is found to be similar as reported by Simian
I ended up building this report via Groovy, which provided an excellent infrastructure for easily parsing the reports from various tools and building a resulting XML document. For example, via Groovy’s hip MarkupBuilder, creating the resulting XML is as easy as writing:
String generateReport(){
def writer = new StringWriter()
def builder = new MarkupBuilder(writer)
builder.analysis() {
project(name:"${projectName}"){
build(label:"${buildLabel}",time:"${buildTime}"){
code_size(){
classes(this.getClasses())
loc(this.getLOC())
}
tests(){
tests_run(this.getTestsRun())
failures(this.getFailures())
branch_coverage(this.getBranchCoverage())
line_coverage(this.getLineCoverage())
}
static_analysis(){
pmd_violations(this.getPMDViolationCount())
findbugs_violations(this.getFindBugsViolationCount())
}
code_metrics(){
code_duplication(this.getCodeDuplication())
max_complexity(this.getMaxComplexity())
max_ca(this.getMaxCa())
max_ce(this.getMaxCe())
}
}
}
}
return writer.toString()
}
Of course, all the parsing is done else where; what’s more, Groovy’s easy Ant integration proved to facilitate using the newly created reporting application with ease. For example, the Groovy application which builds the report is a jar file that other projects then utilize as follows:
<target name="dashboard" depends="groovy-init,all-inspect">
<groovy classpathref="build.classpath">
import org.discoblog.merlin.metrics.report.dashboard.Dashboard
import org.discoblog.merlin.metrics.report.dashboard.tools.ToolProperties
def fullpath = "${properties.basedir}/${properties.defaulttargetdir}"
def dashboard = new Dashboard(projectName:"${pname}",
buildLabel:"${label}", buildTime:"${new Date()}")
//tool properties initialization and subsequent
//adding to dashboard instance...
def dashpath = "${fullpath}/dashboard.xml"
new File(dashpath).write(dashboard.generateReport())
ant.xslt(in:dashpath,
out:"${properties.defaulttargetdir}/dashboard.html",
style:"${properties.defaulttargetdir}/lib/report-style.xsl")
</groovy>
</target>
As you can see from the code above, Groovy marries nicely with Ant and the resulting task (which of course relies on all the previously mentioned 7 reports are run) builds an HTML file like the one displayed above.
Now when a full build is run, rather than sifting through 7 different reports, I can simply examine one report and determine for myself if I’d like to dig deeper (say for instance, there is a test failure or coverage dropped from my previous examination). Hopefully this little report will relieve that jive turkey report overload syndrome and help people in making sense of code metrics. Neat-o, man!
| Related odds and ends | ||
|---|---|---|
12 comments Saturday 27 Jan 2007 | Build Process, Code Metrics, Groovy
12 Responses to “Groovy’s the elixir for report overload syndrome”
Totally disco! this is somewhat linked to what I tried to accomplish with JWatson (still in a very early stage), but mixing Groovy to do the mash-up work… that is genious.
Andres– tell us more about JWatson! What is it?
There is at least one commercial dashboard that automates collection and integration of metrics from multiple tools and sources: Enerjy CQ2. I recently reviewed it in InfoWorld and was failry impressed. It’s a cool tool and, of course, keeps a repository of the metrics so you can see trends over time in addition to the current time slice.
Nice hack! It shows exactly what Groovy is great at.
Andrew– yeah, CQ2 appears to be a slick tool– metrics trending is quite valuable. The tool’s ability to montior KPAs is interesting. I’m also familiar with another tool– Borland’s Gauntlet, which appears to be similar in some ways to CQ2.
Andy, JWatson is the project I mentioned a while ago at my blog (http://jroller.com/page/aalmiray?entry=a_code_metrics_dashboard), the idea is to mix and match several metrics with scm data and get and overall view or your project’s health, think of it like fisheye + cobertura + static analisys. I have rounded up PMD, JDepend and JavaNcss so far (run and process reports) and FindBugs (process report only). Some plugins are not that easy to setup for internal running (FindBugs and Lint4J) so I’m moving at a slow rate. I definitely think that throwing Groovy into the mix is an excellent idea.
[...] Currying maximum favor with Groovy As a side project to building a small build results dashboard with Groovy, I found myself writing a copasetic application, which analyzed a code base using JDepend’s programmatic API. In both instances, I found myself needing to determine the maximum numeric value in a collection. Now, in Groovy (and in any language, for that matter), there is a cornucopia of ways to obtain a maximum value– either through brute force logic or more creatively hip mechanisms. [...]
Andy,
I’ve been jealous of Maven 1′s Dashboard plugin since someone showed it to me long ago, and have since dreamed of how that could be done for Ant. It looks like our dashboard is yet another example of things that Groovy opens the doors to. Very slick!
I haven’t really used Groovy yet, but you’ve whet my appetite. I’ve got a small framework of Ant scripts, that I call Glean, that drives a set of open-source code feedback tools. My goal with the first incarnation was to make something that teams could easily drop into their current build system and quickly get an extensible feedback system going. One of the things on my list to add to it later version is a summary of the results, and I haven’t had any big inspirations. When I saw your description here, though, my ears perked up.
I’m wondering, then, if your dashboard implementation is something that you are considering sharing (and I completely understand if you aren’t). If not, I’m wondering if you could share what your experience was in building it – hard, easy, frustrating, quick? Were there any particularly useful aspects of Groovy that helped? (besides the points you already shared – thanks much for those as well)
In any event, it looks very cool, and I’m glad you at least planted the seed of possibilities in some of us.
Thanks,
John
John- you bet! I was going to dontate the code to Quality Labs. I’d love to see how we can put this stuff and Glean together. Thoughts?
Andy,
My first thought would be to see how this might fit in to Glean as just another tool, albeit something of a meta-tool; the fit might not be perfect, since this would have to run after the other tools (ordering is not important for any of the other tools), but I would like to arrange it so that people could choose whether or not to include this just as they do any of the other tools.
All of the XML data from the other tools would be there already, so with some property values to point at the different locations to slurp it from, I would think it could fit in pretty well. Each of the tools in Glean has their own script, so the dashboard Ant script would live by itself too; the snippet you’ve got above would be the heart of it, and the rest would just be property setup.
I could also imagine allowing users to define some threshold settings for the different tools. These could either be interpreted in the XSLT when you generate the dashboard page, or possibly by the Groovy app. Either way, it would be one way to provide green/red binary feedback on the values as well.
John
[...] Sketching complexity with Groovy Not long ago, I wrote up a nifty dashboarding application, Ã la Groovy, in an effort to abate the visual pain associated with report overload syndrome. These kinds of applications are perfect for languages like Groovy as you can knock them out in a matter of hours (including a test suite to verify kosherness). [...]
[...] In that same theme of getting information out of the collective results, I’m still looking for good ways to put together an “at a glance” summary of the different metrics. Andy Glover has a start on something like this built in Groovy, and I think something like that could plug in to Glean pretty easily. I’m also going to look at XRadar and Hackystat, but at first glance those are a bit more involved to set up. [...]
[...] I’ve wanted to have an all-in-one-page view of a project’s metrics for years now, ever since seeing the Maven 1.x dashboard plugin. And now, thanks to the inspiration from a blog post by Andy Glover and the leverage of a language like Groovy, I’ve managed to pull one together, and have added it to the latest release of Glean. [...]