Code Coverage with Flex - creating EMMA formatted reports
Over the last few months I have adopted Hudson as my build machine of choice as it is just so easy to setup and administer. Another thing I really like is being able to watch the trend of the number of tests in my test harness over time. It's not the best metric, but it does act as a reasonable motivator.
A slightly less crude metric is code coverage, which measures the amount of an application that gets exercised when it's run. FlexCover is a very cool tool for this and props to my colleague - Alex Uhlmann and Joe Berkowitz of Allurent for the great work they've done. There is a great UI for exploring code coverage in detail and it can also export xml formatted reports on coverage.
The thing is, I want to be able to track this coverage over time in Hudson, just like I can with the number of tests. I achieved this by extending FlexCover to output EMMA formatted reports...
So as I was starting to look at this, I could see one of three paths:
- Create a Hudson plugin to consume flexCover's report format
- Add some sort of XSLT transform to my build process
- Modify FlexCover to be able to output a report format that Hudson understands
Creating a plugin for Hudson certainly seemed like a possibility, but the plugins for EMMA and Cobertura were already there and stable, so it seemed like it would be much simpler to try and create a report in a format one of these plugins would understand. Creating an XSLT would work for this, but I'm not an XML guroid, so I figured I'd go down the route of the simplest thing that would work for me and hack extend flexcover to be able to output EMMA formatted reports.
As it turns out this was a pretty simple job, and after a few hours work I managed to create a patch that allows you to output .cvr or EMMA reports with an additional commandline argument. Alex, Joe and I have talked and although my way of doing it works, it is not designed to be scalable or modular to support more formats. They are incorporating the patch, but will probably rework it with these things in mind. They are super busy guys and it wont happen quickly; if you want to in the meantime you can grab the patch from here, but bear in mind it is offered with absolutely no warranties or support :)
Installation
First thing you need to do is checkout the FlexCover code from Google code (http://flexcover.googlecode.com/svn/trunk/CoverageViewer) and patch it with the file attatched. The patch is created from the 'com' level down. To do this, you can right click the file from within Eclipse and select 'Team>Apply Patch...' Follow the instructions and you should be good to go...Usage
In order to write out the coverage report file with the coverage viewer, you need to supply the commandline argument:-output /full/path/to/my/flexcoverreport.xml
This functionality is unchanged, but now you can also specify an EMMA formatted file using the following:
-emma-report /full/path/to/my/EmmaReportName.xml
And that's about it! My next post covers the challenges I had integrating this into my build process, and how I overcame them...
