Build Process

Gant to Ant automagically

On more than one occasion, I’ve been asked by various hip developers if there was a conversion script for transforming existing Ant build files into Gant build files. Russel Winder, the copasetic guru behind Gant updated the Gant website with a link to a trippin’ script (dubbed ant2gant.groovy) that’ll do just that, baby!

I tested it out with a normal Ant build file and it worked like a charm– just make sure you have in your classpath:

  • ant.jar (I used 1.7.0)
  • ant-launcher.jar (I used 1.7.0 as well)
  • commons-cli.jar (I used 1.1)

Also too, my Ant build.xml file had a path element living outside of a target like so:

<path id="build.classpath">
 <fileset dir="lib">
  <include name="**/*.jar"/>
 </fileset>
</path>

This was causing the script (really Ant’s Project object) to blow up; consequently, I moved it into an initializing target and things worked copasetically from there.

If you haven’t tried Gant yet and you find yourself beating your head against the wall when it comes to adding behavior to Ant (such as looping, etc), then give your head a rest and take a look at Gant– this is a smokin’ tool that adds a refreshing level of flexibility and expressiveness to Ant (using Groovy, man!).

Chaffin’ over Groovy with Scott Davis

I had the pleasure of a buzz session with my hip friend Scott Davis recently regarding Groovy and the upcoming Groovy/Grails Experience. Scott is a entertaining guy to chat with and I had a lot of fun describing BDD with easyb and the joy of using Gant. Check out the hip podcast if you have a few spare minutes and don’t forget to show up to the conference later this month, baby!

Joining the bash at CodeMash, man

As if seeing a groovy gal shave a dude’s head and hanging out with copasetic cats like my hip friends Neal Ford, Brian Sam-Bodden, and Chris Judd weren’t enough reasons to head out to Ohio for CodeMash, I’m pumped to have been invited to present two sessions and one open spaces event.

I’ll be speaking about BDD, Groovy, and a Martin Fowler inspired subject regarding the future of build languages. My BDD talk will obviously focus on how BDD can drive development more effectively than TDD and I’ll demonstrate some JBehave, RSpec, and a Groovy inspired RSpec knock-off. My Groovy session is a veritable crash course on using Groovy quickly– 101 style, baby. Lastly, the build languages openspaces session is an open discussion regarding Ant style (i.e. XML driven platforms like NAnt and MSBuild) build platforms versus more expressive platforms like Rake, Raven, and Gant, for example.

With sessions on Castle, Dojo, Grails, Rails, Scala, CI, and myriad other hip subjects, CodeMash looks like it is going to be a blast, baby– if you’re going to be there, let me know!

What’s after Ant?

At a copasetic CITCON session in Dallas recently, Martin Fowler mused on the “future of build languages” and suggested that due to Ant’s inability to be richly expressive (think conditionals, looping, etc) that future build platforms will be modeled as native domain specific languages much like Rake is in the Ruby world (or SCons is in the Python world). Note, Ant is a DSL, but it is expressed in XML not Java, man.

Because it’s my bag, I largely agree with Martin, however, given Ant’s deep penetration within the Java market, compared to Maven (which, I think is one step closer to a Rake-like build platform for Java) I wonder if the Rake model can come to fruition and obtain any reasonable mindshare. Sure, there are hip projects underway like Raven (which combined forces with JRake recently), which does a number of useful things such as compilation, testing, etc, but ultimately, as Ant is the de facto build platform for Java, practically every tool out there bundles an Ant task. If you want to use Raven and would like to generate a report with JavaNCSS, for example, you might have to write some custom Ruby code to fire the JavaNCSS process off. Hopefully, the increased velocity of development on JRuby will alleviate this issue as it appears JRake makes use of Ant tasks.

It seems to me that if a more DSL-like platform is going to gain any mindshare within the Java community, it’ll have to take advantage of the rich infrastructure present within Ant (Maven did this though and hasn’t overtaken Ant by any means). One such project is Groovy’s Gant, which enables you to specify builds via Groovy rather than XML. Groovy provides expressiveness and is native to the JVM; however, given the history of Java build platforms, I suppose Gant’s adoption will lag behind Maven by a long shot, man.

Interestingly, a few individuals suggested that some of the issues associated with Ant were not necessarily related to Ant, but to people’s usage of the tool. This made me remember an excellent article published a number of years ago that largely still holds true: “Top 15 Ant Best Practices.”

Ultimately, Ant is the de facto standard for building Java applications. Like it or not, it works for 90% of those that attempt to employ it. There will be innovative and clearly more flexible build platforms that’ll pop onto the scene, but displacing Ant is much like thinking that Macs or Linux will overtake Windows installations (in the near future). Displacing these kinds of market leaders takes seismic paradigm shifts (like we are currently seeing with Disco music).

What’s after Ant? Well, we’ve seen what’s next: it’s Rake (or Raven or Gant), but for Java, the vast majority of the community doesn’t appear to care. That’s so establishment.

An interview with Cargo’s Vincent Massol

In the past, I found myself on a number of different copasetic occasions struggling with higher level test repeatability. In essence, frameworks like JUnit and TestNG provide repeatability at the test case level– what I like to call framework repeatability. But logical repeatability, that is, the assumptions regarding the environment in which a hip test is to run in, is completely up to you, the test case author. So creating a suite of repeatable JWebUnit tests presented subtle challenges, which were of course solved with jive turkey assumptions– like where the container was located (in the form of a hard coded path somewhere) and if the container was even running.

Then one day, I stumbled upon Cargo, an open source project aimed at automating container management in a generic way so that I could use the same mechanism to start and deploy a war file to Tomcat as I could with WebLogic. Since then, because it’s my bag, I’ve been a Cargo fanatic, spreading the good word of logical repeatability, which facilitated an introduction to Vincent Massol, the founder of Cargo.

Vincent is no stranger to the Java world, having written Manning’s smokin’ “JUnit in Action” and O’Reilly’s “Maven: A Developer’s Notebook“; furthermore, Vincent has created a host of plug-in’s for both Maven and Maven 2. His long running blog, the Vincent Massol Think Tank has been a great read for years as well. Now, he’s working for XPertNet the company behind XWiki, the Java open source second generation wiki.

I recently had the opportunity to ask a few questions of Vincent regarding Cargo:

The Disco Blog: What’s left for Cargo before is 1.0, man?

Vincent: We’re close to a 1.0 release (we’re gearing up for the 0.9 release right now). We’d like to have a stable API for 1.0. In the past 2 Cargo releases we’ve introduced new concepts: runtime containers, remote deployments and embedded containers. These concepts are now in place and the API is now stable. Once we release a 1.0 we’ll need to go through deprecation cycles if we want to modify the API. In term of features for 1.0 we need:

  • More container implementations (e.g. WebSphere and Glassfish) and new versions of existing containers (Geronimo 1.1, WebLogic 9.x, etc).
  • JSR88 support for deployments. We have started this a long time ago but we haven’t finished it as the person interested in it has not been active. We need to resume the work as more and more containers are becoming JSR88-compliant.
  • More remote deployers. We already have deployers for almost all containers but we now need to have more remote ones (i.e. the ability to deploy to a container running on some remote machine).

The good news is that we’re getting more and more contributions from the community.

The Disco Blog: What are your thoughts on JSR88? Did it provide an influence on Cargo or vice versa? Just sock it to me, man!

Vincent: JSR88 standardizes deployments into J2EE containers. Cargo is about starting, stopping containers, configuring them and deploying to them. This is a superset to what JSR88 provides. We have started developing a JSR88 deployer a year ago. However and unfortunately, the committer who started this work is no longer active so it’s never been completely finished (hint: if anyone is interested in helping out please contact us!). However, this is one of the must have features that we want for the Cargo 1.0 release. That said, we also need to continue supporting the container-specific deployers we have, as not all containers support JSR88 and Cargo still needs to be able to deploy to any version of containers, even old ones. For the end user, this is all transparent as users are using the Cargo Deployer interface. At one point in the far future we may want to deprecate our Deployer interface and instead directly expose the JSR88 one but the time is not ripe yet.

The Disco Blog: As of now, Cargo lacks WebSphere and Glassfish containers– which containers do you currently support and are there any others still in need?

Vincent: I think we have quite a lot of them already (Tomcat, Resin, JBoss, WebLogic, Orion, OC4J, Geronimo, Jetty). There are 2 important ones missing that come to mind:

  • WebSphere
  • Glassfish

Then of course there are lots of others that we would also like to have, such as JonAs, OpenEJB, etc. We need all the help from the community here. That’s probably one of the biggest challenges of Cargo: to be able to get experts in the different containers to provide implementation and support for them in the long run. We’re very open and keen to get new committers for scaling up Cargo.

The Disco Blog: Let it all hang out, man– which containers have the strongest support in Cargo?

Vincent: This is driven by the community. Today the most used container is probably Tomcat with JBoss coming second. Jetty is very close behind. That said, we have very good support for Orion and Resin too. We’re currently improving our WebLogic support by adding deployers and implementing support for WebLogic 9.x.

The Disco Blog: When we first traded some hip emails on Cargo, you stated that the Maven 2 plug-in for Cargo has more features than the Ant task– so what features does the Maven 2 plug-in have that the Ant task doesn’t?

Vincent: Right now the Cargo Ant task does not support Deployers. The Maven2 plug-in also has some additional nice features that cannot exist in Ant (because Ant doesn’t have the notion of project structure) such as automatic deployment of J2EE modules (if the project is of type EAR, WAR, EJB, etc) then the Maven2 plug-in automatically deploys that artifact to the executing container. There are also lots of configuration elements that have default values in the Maven2 plug-in that do not exist in Ant.

The Disco Blog: What is your preferred way to interface with Cargo– Maven, Java API or Ant?

Vincent: The primary interface is the Java API. This is what Cargo is about. Then we have extensions which are users of the Cargo Java API and out of these extensions the Maven2 one is currently the most advanced.

Thank you very much, Vincent for your time and valuable insights into the Cargo project– good luck, man!

Groovy’s the elixir for report overload syndrome

Not long ago, I posted a poll regarding code metrics in which the majority of votes settled on two, not too surprising, copasetic points:

  1. Some people are not sure what the data means
  2. Others aren’t sure which tools to use to obtain the data

Issue #1 spawned a posting a few weeks back regarding the meaning of code metrics; however, issue #2 got me thinking– there are quite a few different tools out there that gather diverse metrics, yet there are few opportunities to effectively view them. For instance, for Java projects, I often find myself recommending people use JavaNCSS, Cobertura, and PMD to name a few hip tools. In fact, in all, one can categorize various tools’ outputs into seven metric types:

This list, by the way, is composed of Paul Duvall’s Big Five code analysis areas– because it’s my bag, I’ve added test results and code size.

But looking at the list above reveals over six different reports, each with different formats and variations on how data is visually displayed. This cornucopia of data often leads to report overload syndrome, in which, because of information overload, the data has the tendency to become ignored (not much unlike the lack of an Oscar for the ever so hip Saturday Night Fever).

As I was sitting on a plane recently I found myself looking for an easy way to disseminate the valuable data from the categories above in an effective manner. I ended up with a design of a small table that displays summary data from the various tools and which provides links to the actual tool’s report for a more in depth analysis, should one feel it warranted.

dashboard

As you can see from the image above, the table summarizes the output of seven tools used in a typical build:

  • Test results from JUnit or TestNG
  • JavaNCSS’s count of classes and lines of code as well as the maximum cyclomatic complexity found in an individual method
  • Cobertura’s line and branch coverage
  • The count of FindBugs violations as well as PMD’s count
  • JDepend’s maximum reported Afferent and Efferent coupling
  • The amount of code which is found to be similar as reported by Simian

I ended up building this report via Groovy, which provided an excellent infrastructure for easily parsing the reports from various tools and building a resulting XML document. For example, via Groovy’s hip MarkupBuilder, creating the resulting XML is as easy as writing:

String generateReport(){
 def writer = new StringWriter()
 def builder = new MarkupBuilder(writer)

 builder.analysis() {
  project(name:"${projectName}"){
    build(label:"${buildLabel}",time:"${buildTime}"){
      code_size(){
          classes(this.getClasses())
          loc(this.getLOC())
      }
      tests(){
          tests_run(this.getTestsRun())
          failures(this.getFailures())
          branch_coverage(this.getBranchCoverage())
          line_coverage(this.getLineCoverage())
      }
      static_analysis(){
          pmd_violations(this.getPMDViolationCount())
          findbugs_violations(this.getFindBugsViolationCount())
      }
      code_metrics(){
          code_duplication(this.getCodeDuplication())
          max_complexity(this.getMaxComplexity())
          max_ca(this.getMaxCa())
          max_ce(this.getMaxCe())
     }
    }
   }
  }
  return writer.toString()
}

Of course, all the parsing is done else where; what’s more, Groovy’s easy Ant integration proved to facilitate using the newly created reporting application with ease. For example, the Groovy application which builds the report is a jar file that other projects then utilize as follows:

<target name="dashboard" depends="groovy-init,all-inspect">
 <groovy classpathref="build.classpath">
  import org.discoblog.merlin.metrics.report.dashboard.Dashboard
  import org.discoblog.merlin.metrics.report.dashboard.tools.ToolProperties

  def fullpath = "${properties.basedir}/${properties.defaulttargetdir}"

  def dashboard = new Dashboard(projectName:"${pname}",
     buildLabel:"${label}", buildTime:"${new Date()}")

  //tool properties initialization and subsequent
  //adding to dashboard instance...

  def dashpath = "${fullpath}/dashboard.xml"

  new File(dashpath).write(dashboard.generateReport())

  ant.xslt(in:dashpath,
    out:"${properties.defaulttargetdir}/dashboard.html",
    style:"${properties.defaulttargetdir}/lib/report-style.xsl")
 </groovy>
</target>

As you can see from the code above, Groovy marries nicely with Ant and the resulting task (which of course relies on all the previously mentioned 7 reports are run) builds an HTML file like the one displayed above.

Now when a full build is run, rather than sifting through 7 different reports, I can simply examine one report and determine for myself if I’d like to dig deeper (say for instance, there is a test failure or coverage dropped from my previous examination). Hopefully this little report will relieve that jive turkey report overload syndrome and help people in making sense of code metrics. Neat-o, man!

Refactoring in NAnt generic-ness

I recently needed to create a copasetic NAnt script that handled database schema management– i.e. the script would create a database and populate it with seed data. Such a process is central to the notion of Continuous Database Integration (which happens to be the subject of chapter 5 in Addison Wesley’s forthcoming book entitled “Continuous Integration: Improving Software Quality and Reducing Risk“).

Using NAntContrib’s sql task, I proceeded to create two hip targets– one which ran a DDL script that dropped any existing tables and then re-created then. The second target ran a DML script, which then proceeded to populate the database schema with data. This is essentially an up front process that enables one to, say, deploy a fully functioning application for testing, in an automated fashion from within a build process. Lastly, I created an additional boss task, dubbed database-prepare, which called the first two tasks and acted as a handy facade.

Once completed, I was essentially left with a NAnt script that screamed in the face of the DRY principle as I had two tasks that did essentially the same thing. For example, below is a snippet of the offending build file:

<target name="database-prepare">
 <call target="database-create"/>
 <call target="database-load"/>
</target>

<target name="database-create">
 <echo message="Creating database definition with ${data-definitions}"/>
 <sql connstring="${project.db.conn}"
     delimiter=";" delimstyle="Normal"
     print="true" source="${data-definitions}"/>
</target>

<target name="database-load">
 <echo message="Loading database with data found in ${data-load}"/>
 <sql connstring="${project.db.conn}"
     delimiter=";" delimstyle="Normal"
     print="true" source="${data-load}"/>
</target>

At this point, because it’s my bag, I found myself feeling somewhat disquieted (of course, in a disco sort of way) longing for a more expressive form of build scripting, say with something like Boo. But I found myself not needing a script task, for example, but a more rich way of structuring a build– like what one finds in Rake. With more expressive build platforms (that don’t rely on XML) one can create reusable functions, for example, which is exactly what I wanted– a generic sql task that I could then pass in the desired load file at build time depending on the command.

In the end, rather than rewrite the entire build, I chose to force target reuse in NAnt via the property task. While not perfect, it works– the build file has a generic sql target that relies on a smokin’ property being set– if the property isn’t set, the target will fail. The way the build is structured, the only way to cause a failure is to call the generic target directly (to my knowledge, there isn’t a way to explicitly deny calling a target directly– private targets anyone?). I chose to name the target with a pythonic underscore to signify this target’s private-ness.

The API of the build is essentially still the same– there are three public targets, which correspondingly initialize the property to the proper DDL or DML file and then call the generic sql target as follows:

<property name="data-load" value="./sql/data-load.sql" />
<property name="data-definitions" value="./sql/data-definitions.sql" />

<target name="database-create">
 <property name="data-file" dynamic="true" value="${data-definitions}"/>
 <call target="_database-do"/>
</target>

<target name="database-load">
 <property name="data-file" dynamic="true" value="${data-load}"/>
 <call target="_database-do"/>
</target>

<target name="_database-do">
 <echo message="loading ${data-file}"/>
 <sql connstring="${project.db.conn}"
     delimiter=";" delimstyle="Normal"
     print="true" source="${data-file}"/>
</target>

<target name="database-prepare">
 <call target="database-create"/>
 <call target="database-load"/>
</target>

While not perfect, it certainly is more in line with what I’d do if I were coding this logic in C#, for example. A build file (or files) is one of the most important assets a project has– without an effective build, no matter how many tests you write or how perfect the code is, customers will have a hard time receiving working, reliable applications. So the next time you find yourself wincing at your build script, take the time to refactor it, man!

The binary dependency boogy

Sooner or later, every software project ends up relying on some binary module, which could could be a third-party library or another project in compiled form. In the Java world, these are jar files (remember back in the Age of Aquarius when they were zip files too?!) and on the Microsoft platform they’re dlls.

Given that it’s pretty much a universal truth that eventually you’ll have to come up with a binary dependency management scheme, I’m amazed with the myriad different ways I’ve seen this handled. Two of the most common techniques I’ve seen are:

  • Place all libraries on a shared drive
  • Place all libraries in a lib folder underneath the project’s root

Of course, these strategies work; however, they aren’t optimal. Placing all libraries on a shared drive becomes a maintenance nightmare if they are haphazardly placed there– like in one giant directory. Then of course, you have the issue with drive mapping, which works fine if everyone’s on the same OS.

Placing libraries in a specific project folder (and of course, adding them to an SCM) is a good strategy in that a project’s dependencies are managed within the project itself. If you end up having multiple projects that have the same dependencies; however, you end up with the same jar file, for example, copied throughout an SCM, which has always seemed inelegant to me.

I, instead, prefer a binary dependency management mechanism spearheaded by the Maven project, which can be emulated in Ant, NAnt, and any other build platform that supports HTTP gets. Using this scheme, dependencies are placed on a web server and are versioned via a naming scheme. During the build process, required libraries are downloaded to the local machine and are included in the classpath for compilation, runtime execution, etc.

The beauty here is the naming scheme. For example, say I’m building a project which uses JUnit for its test framework; consequently, to compile and run resulting tests, the build process requires JUnit’s jar file in the classpath. If my project requires version 3.8.1, then I can create a few properties to represent the dependency as follows:

<property name="junit-jar" value= "junit-3.8.1.jar" />

Then I need a disco place to download it from– I can either set up my own web server for the project, or use a publically available one, such as ibiblio.org, which in that case, the download URL would be the base: http://www.ibiblio.org/maven/junit/jars/ and then the jar file: junit-3.8.1.jar.

I also need to specify a place to download dependencies to, which in my case is set via a property:

<property name="libdir" value="target/lib" />

Note too, I can have multiple repositories from which to obtain dependencies from, so I can create a repository property as well. Putting it all together yields a get that looks like this:

<get dest="${libdir}/${junit-jar}" usetimestamp="true"
       ignoreerrors="true"
       src= "${repo-base}/junit/jars/${junit-jar}" /> 

Creating a classpath is as easy as including all the jars in the target/lib directory as I’ve done below.

<path id="build.classpath">
  <fileset dir="${libdir}">
   <include name="**/*.jar">
   </include>
  </fileset>
</path>

Of course, targets which compile code or run tests, would then depend on the dependencies being present; accordingly, they can specify a depends clause which links back to a target that performs a series of gets.

<target name="compile" depends="get-deps">....</target>

The get-deps target also has the hip ability to skip downloads should the libraries already be in place; therefore, a noget property is available as shown below:

<target name="get-deps" unless="noget" depends="init">...</target>

I should add that there are more sophisticated mechanisms available for Ant which handle dependency management– Ivy is one such project that is probably the most feature rich. Both Ivy and Maven2 support the notion of transitive dependencies, which is way disco and limits a lot of the grunt work when specifying dependencies. If you’ve ever done this with something like Hibernate or Spring, you’ll know what I mean.

What I’ve shown here, by the way, is applicable in NAnt as well using its get task.

The next time you find yourself unable to build because the link to the share drive is down or you are unsure what exact version of a library is required for compilation, consider using the Maven scheme for dependency management– it’ll free up some copasetic time for more disco dancing.

Three most important things that improve code quality

When considering code quality, there are three aspects that have the most copasetic effect. Of course, there are other techniques and practices that positively affect code quality; however, it’s been my bag that these three, listed in their order of importance, offer the biggest bang for the buck.

The first aspect that affects code quality is developer testing. It’s been said that “disciplined personal practices can reduce defect introduction rates by up to 75 percent.” Surely, writing a developer test is a circumspect process that helps verify program correctness! And program correctness is directly related to quality, man.

At its core, a developer test verifies a portion of code in an isolated manner that can’t be achieved through latter cycle neat-o functional style testing. From a reliability standpoint, in linear systems (which are substantially less complex than software systems, which are non-linear), “the product of the reliability of each of the system’s components” equals the reliability of the overall system. Moreover:

“If you wanted to build a software application that had an SLA or QLA of 100% (or close) you’d absolutely have to ensure reliability at the individual object level. In fact, if you can’t ensure and measure reliability at the lowest level, you can’t possibly do that at the system level.”

Developer testing, you see, is paramount in guaranteeing code quality as it verifies program correctness at the lowest possible level which is unattainable via square functional testing.

The second most important aspect which facilitates improving code quality is developer testing. White box tests offer a confidence cushion that effectively facilitates refactoring, which in my opinion is “the [hip] act of improving code that has already been improved.” Without a test case, refactoring code becomes a dangerous proposition– if you can’t prove the code worked before you improved it, how can you verify you haven’t broken it after touching it? Hence, while you may feel that improving code quality starts (or ends) with refactoring, I suggest that without developer testing, refactoring is a non-starter.

The third most important aspect which facilitates improving code quality is developer testing. Code is entropic– meaning that:

“A software system that undergoes continuous change, such as having new functionality added to its original design, will eventually become more complex and can become disorganized as it grows, losing its original design structure.”

The only way to combat this natural tendency towards code clutter is through developer tests, which act as the only form of documentation that clearly states the intent of code and isn’t affected by humans forgetting to update disconnected documentation. If the code under test changes in any way, the tests will figure it out by failing, in which case, the smokin’ tests are updated to reflect the new behavioral contract. Rarely have I seen disconnected documentation, in the form of comments or design documents, stay current as effortlessly. Clearly documenting a program’s intent at all levels is surely a compelling way to repel code entropy.

Indeed, the three most important things which improve code quality are developer testing, developer testing, and developer testing, because developer tests facilitate:

  • software reliability, at the lowest possible level
  • code modifications with confidence
  • keeping code entropy at bay through documentation that endures

Software metrics, automated builds, quality focused processes, people, etc all positively affect code quality (and are, of course, quite important), but it all starts with developer tests. Dig it?

Going Dutch on automation

Automated builds shouldn’t be the hidden magic of hip development teams– if builds are truly automated (via Ant, NAnt, etc), then everyone should be able to produce validated binaries, right? In organizations that have CM teams, these gurus usually do utilize the same development build process to promote applications; however, for organizations that don’t have the luxury of gate keepers, how are builds promoted?

I’ve run across different processes for binary promotion usually involving a manual hand-off between development and QA; however, I’ve rarely seen these two copasetic organizations share the same build process. If development has an automated repeatable build process that produces verified binaries (all developer tests pass, for example), then QA should be able to run the same build.

In this scenario, the hand-off is a CM label, which development has created; accordingly, the process, at a high level, works as follows:

  1. The development team decides they have a release candidate.
  2. They label the assets in the CM system according to some naming scheme (i.e. major.minor like 3.21).
  3. This mellow label is communicated to the QA team, who runs a build that checks out the labeled code, runs all checks (like tests, static analysis, etc) and produces a binary (or set of binaries) ready for functional testing. Even the deployment of the binary assets can be automated.

Note how this process is automated, man– even the communication of a candidate label is automate-able.

For example, imagine that development has indicated that version 1.2 of project X is ready for functional QA testing. QA receives this notice, via email or RSS or in a meeting, and runs a build passing in the label identification. Then labeled code is checked out and built. Functional testing heaven is then possible, baby!

Because it’s my bag, check out a simple, yet totally disco, Dutch automation process in this 30 second movie. Neat-o!