Code Coverage with Flex - a headless agent for CI builds
July 28, 2009
In my last blog post I gave details of how I user the modified coverage viewer in an automated build to follow the trend of code coverage over time. The trouble with this approach was that there was a problem either with the localConnection or the code that uses it and there was a wide variance of the values being reported. This post shows you how I fixed it by creating a headless coverage reporter that you can drop into your test harness and remove the need for a second application altogether.
In order to try and do this I decided to use as much of the code as possible that the FlexCover guys had written, and only change what I needed to get it working.Once I had it up and running I could worry about making it faster / leaner / more neatly coded...
The approach
So here's the basic plan:- Swap out the default localConnection reporting mechanism for one that will pick and choose depending on environment the app is running in
- Create a headless agent that will pull the coverage data collection and reporting side of the coverageViewer into the app
- Use commandline options to swap between the headless agent and local connection agent
- Default to using the localConnnection agent
How it works
When your app compiles using the instrumented SDK created for using FlexCover, your application makes use of a class called CoverageManager. This manager is a patch that allows you to plug in a custom coverage agent for use in your app. By default it uses a class called LocalConnectionCoverageAgent which broadcasts coverage metadata to the coverage viewer application. What my patch does is get in and allow you to use a different, headless agent. To do this simply call during the preinitialize event of your main application:
private function injectAgent():void{
CoverageManager.agent = new CoverageAgentSwitch();
}
As a quick test, run your app as you had previously and confirm it still works by sending data over a localConnection to the CoverageViewer. This is expected as by default, the switch will create a LocalConnectionCoverageAgent. In order to use the headless agent you need to set a few commandline properties. First you need to tell the switch that you want to use the headless agent. Then the headless agent needs to know where it's getting its metadata from, where it should output the coverage report. The commandline options are:
-coverage-agent-type=headless Headless is only option, anything else will default to localConnectionAgent
-coverage-metadata='/full/path/to/coverage/metadata.cvm'
-coverage-output='/full/path/to/coverage/reportInOriginalFormat.cvr'
-emma-report='/full/path/to/coverage/reportInEmmaFormat.xml'
Note that if you don't specify the metadata path and at least one of the report formats, the headless agent will log errors but otherwise fail silently.
In terms of the edits I made to get it working, that is pretty much it. The headlessCoverageAgent swc is probably larger than it needs to be and is definitely grossly inefficient. I will update this soon to improve this, but right now I only have time to get this post up.
Obviously you will have to change your buildscript to pass these new commandline parameters in to the testharness when you run it. If there is any interest, I'll post my updated build script and testharness modifications that dispense with the log parser altogether and make for a more repeatable build script.
The Result
My build process is now much quicker because I don't have to wait to be sure that the coverage viewer has initialized. It's also completely stable and the coverage trend has been a great motivator for the team.
Coverage trend without crazy variance found using the external viewer
