Eyefodder

In this, the final entry on CI with Flex, I look at an extra change we made to the ASUnit sources, take a look at what CI has done for us, and finally let you in on what we are looking at next...

Execution Time

When we got CI up and running I started looking more closely at the information that JUnit gives out versus the information that ASUnit gives. The only omission I found was that in JUnit you get feedback on the execution time of a particular assert. I looked at how we might implement this in ASUnit, and made a slight chage to the sourcecode that gives us an execution time. The problem here is that execution time is displayed per assert (in Cruise Control), but calculated per function, so its not completely accurate. What it will do however is help to nail down code that executes slowly.. in order to get this to work, you will need to find the

runMethod()

function in

TestCase

, change it and add the method

appendExecutionTime

like this:

private function runMethod(method:String)
	{
		setUp();
		setCurrentMethod(method);
		var before:Number = getTimer();
		this[method]();
		var elapsed:Number = getTimer() - before;
		appendExecutionTime(elapsed/1000);
		tearDown();
	}
	private function appendExecutionTime(n:Number):Void{
		var list:TestRunner = getTestRunner();
		var max:Number = list.length;
		var cName = getCurrentClassName();
		var cMeth = getCurrentMethod();
		var item:Object;
		for (var i=0;i
How the non-technical team responded to CI
As we had expected, CI increased the visibility of the development process. We expected this to be seen as a good thing, and in many cases it was. We also had a few unexpected results too. Members of the business team would panic when the most recent build was either broken, or had seemed to take a step back. We as a development team had figured that everyone else would realise that as a development machine, sometimes we will break stuff (of course, we fix it as soon as possible...) and that this was no big deal. But the problem was that they hadnt been used to this level of visibility, and it was just a scary thing. A little bit of education down the line and they were more relaxed about it though...
On the plus side, having CI as part of our process did reduce the number of nasty, hard to find bugs in our app. Unfortunately I dont have any hard evidence for this, just empirical, but I think I'm right... 
Next Steps
This is only the beginning of our automated testing experiment. We would like to take this approach further by looking into automating acceptance(functional) tests into our framework. Adobe are working on a functional testing tool to integrate into Mercury QTP. We would like to look at incorporating this into our automation process so that there is an even higher degree of confidence that what the development team offer out to the larger project team works harder better faster and stronger… 
Another thing I am interested in looking at is how to shorten the feedback loop even further. I would like to have our Cruise Control build results page have a simple feedback form in it which will tie into our bug tracking system. This will mean that it is even easier for people to give us feedback on development. We will also be tagging feedback to build numbers, so we can keep track of when a bug was spotted.
There are two things I have been thinking of as weaknesses in our process that could do with being addressed, but I dont really know the answer to yet. The first is unit testing GUI's which I have never seen a satisfactory solution for. We have mitigated this by removed everything but layout from MXML components, so at least we can test business logic, but Im sure there is a better solution out there.
The second thing that has been nagging me is how we go about writing better (read complete) unit tests. The thing is that when doing test-driven-development, its all too easy to think you are too busy to write a test just now, or that you will 'go back and write the tests later'. This is a bit of an arse about tit mentality - you will never get around to writing tests if you dont do it when you write the code. Some gains can be made with self-discipline, but I have been mulling the idea of a code coverage tool for AS. I'm still not sure if this will be useful (or used) but its one of many side-projects spinning wround in my head at the moment...
                              
                              

                                 Posted by Paul BH at 07:22 PM | Permalink
                                 | Comments (1)
                                 
                              	
Tags 
                                  |
                           Flex 
                        |
                           Continuous Integration 
                        |
                           Best Practices 
                        |
                           Test Driven Development 
                         
                              
                           

This is the penultimate installment of the CI saga, and its the last step you will need to take to get your continuous integration suite up and running. The first thing you will need to do is install CruiseControl onto your build monitoring machine. The first thing I did was get it up and running on my machine, and then re-follow my steps on a server.

CruiseControl is an open source build monitoring tool. The good thing about it is that it can be easily configured using an XML file, has plugins to lots of other tools , and has a simple to understand web based interface. I just installed from the binaries, and followed these quickstart instructions. One thing I did have to do that wasnt covered in the instructions is set an environment variable, JAVA_HOME to the location of java on my machine (in my case: C:\Program Files\Java\jdk1.5.0_04). Once you have it up and running with the simple connectfour example, we are ready to get going with our flex project.

Check out the project repository to cruisecontrol's projects folder

The first step is to create a new folder in the ‘projects’ folder of cruisecontrol and check out your project to that folder. This is the copy of the repository that cruisecontrol will be using to run the master build. The name of the folder will be the same as the name displayed by Cruisecontrol for your project.

Copy your required classes to the ant directory of Cruise Control

Remember those ant classes I got you to download and put in the lib folder for your local build? Well, cruise control is going to need them too if its going to build successfully. so, look for the ant folder in cruisecontrol (apache-ant-1.6.5\lib) and put the classes you downloaded in step 2 in there.

Make your build CruiseControl happy

CruiseControl checks at specified intervals to see if any changes have been made to your source code, then it will trigger a build. This is all well and good, but the main thing to consider is that even though cruisecontrol will know to trigger a build if the source code has changed, it does not automatically do an update from the repository. So before we go any further with cruise control, add the following target to your ant build:

  
<target name="update">
   <svn>
     <update dir="${basedir}" />
   </svn>
 </target>

And change your ‘all’ target (again) to look like this:

<target name="all" depends="update,determineTestHarnessSuccess,compileMain"/>

Be aware though that this solution is not ideal. If you change anything the ant script needs before it runs, these will not be up to date (until the next time the ant script runs) for example, your build properties file, or any jar’s referenced and stored in the repository. In practice, this shouldn’t happen too much, and isn’t worth worrying about (too much)…

Configure CruiseControl

The next step is to configure cruise control for your needs... First off, to get somewhere, you will need a place for logs to be put so that CruiseControl can read them and tell you how your app is doing. Make a folder in the logs directory with the same name as the one you made in the projects folder. You will probably want to publish build artifacts too. Basically, when CruiseControl has a successful build (or even an unsuccessful one if you like), it can publish artifacts by copying a file or directory somewhere. This is incredibly useful for tracking the progress of a project, as you get to see a snapshot of your project at each successful build. Go ahead and make a folder in the artifacts folder of cruise control and give it the same name as the folder in projects...

Now we need to write our config XML for the project. This is a relatively simple affair, but will be different according to your needs. Open up config.xml in the cruisecontrol directory and take a look at how its setup for connectfour(the sample project). We setup our build to run every 15 minutes, publish successful builds to the artifacts directory and mail the project team the results of the build, but you can set it up to do pretty much what you like... The configuration reference is invaluable for showing you what you can do with CruiseControl... Below is an outline of how we set our build up:

<cruisecontrol> 
<project name="testFlexProject" buildafterfailed="false">
		<listeners>
			<currentbuildstatuslistener file="logs/${project.name}/status.txt"/>
		</listeners>
		<!-- Bootstrappers are run every time the build runs,
       *before* the modification checks -->
		<bootstrappers>
		   <svnbootstrapper
		        file="projects/${project.name}/build.xml"/>
		</bootstrappers>
		<!-- Defines where cruise looks for changes, to decide
        whether to run the build -->
		<modificationset quietperiod="10">
			<svn localworkingcopy="projects/${project.name}"/>
		</modificationset>
		<!-- Configures the actual build loop, how often and which
        build file/target -->
		<schedule interval="900">
		
			<ant anthome="K:\cruisecontrol\cruisecontrol-bin-2.4.1\apache-ant-1.6.5" buildfile="projects/${project.name}/build.xml" target="all" uselogger="true" usedebug="true"/>
		</schedule>
		<!-- directory to write build logs to -->
		<!-- log logdir="logs/${project.name}"/ -->
		<log>
			<merge dir="projects/${project.name}/log/test-results"/>
		</log>
		<!-- Publishers are run *after* a build completes -->
		<publishers>
			<onsuccess>
				<artifactspublisher dest="artifacts/${project.name}" dir="projects/${project.name}/bin"/>
				<htmlemail mailhost="YOURMAILHOSTHERE" returnaddress="richard.nixon@YOURCOMPANY.com" returnname="Nixon" buildresultsurl="CRUISECONTROLIPADDRESS:8080/buildresults/${project.name}" skipusers="true" xsldir="C:/cruisecontrol/cruisecontrol-bin-2.4.0/webapps/cruisecontrol/xsl" css="C:/cruisecontrol/cruisecontrol-bin-2.4.0/webapps/cruisecontrol/css/cruisecontrolEmail.css">
					<always address="youremailinhere"/>					
				</htmlemail>
			</onsuccess>
			<onfailure>
				<htmlemail mailhost="YOURMAILHOSTHERE" returnaddress="richard.nixon@YOURCOMPANY.com" returnname="Nixon" buildresultsurl="CRUISECONTROLIPADDRESS:8080/buildresults/${project.name}" skipusers="true" xsldir="C:/cruisecontrol/cruisecontrol-bin-2.4.0/webapps/cruisecontrol/xsl" css="C:/cruisecontrol/cruisecontrol-bin-2.4.0/webapps/cruisecontrol/css/cruisecontrolEmail.css">
					<always address="youremailinhere"/>
</htmlemail>
			</onfailure>
		</publishers>
	</project>
</cruisecontrol> 

That is pretty much it – in our set up cruisecontrol checks the repository every 15 minutes for code changes, and if the build succeeds, everyone gets an email pointing them to the latest build results. It has already proved itself in the first few weeks of getting up and running – the project team now know where we are in a project, and know where to get hold of the latest build. On top of this, they can browse through previous builds and see exactly how we’re progressing…

Posted by Paul BH at 06:40 PM | Permalink | Comments (0)
Tags | Flex | Continuous Integration | Best Practices | Test Driven Development

Ok, so far so good. We had covered a lot of ground up to this point, but we were only part of the way there… The next step is to get our test results understood by ANT. Ikezi created a natty python script to parse the output of our flash logfile using some cunning regular expressions. Basically, it looks for a multiple line match of __TRACERUNNEROUTPUTBEGINS__(somestuff in here)__TRACERUNNEROUTPUTENDS__, then parses the match and places the test result (true / false) into a test status text file, and the XML portion into another log file.

Create a new file called flashLogParser.py and paste the following into it:

## ------------------------------------------
#   flashLogParser.py
#
#   Ikezi Kamanu
#   Paul Barnes-Hogget
#
#   February 2nd 2006
#
#   Python Script to parse flash log for
#   unit test results. 
## ------------------------------------------

import re, sys

class LogParser:

    filepath =  None

    file = None

    testOutput = None

    testStatus = None

    testLogOutput = None

    

    

    # Constructor: Receives file path, sets in self

    # ------------------------------------------------

    def __init__(self, pathToLog):

        self.filepath = pathToLog

        self.setTestOutput()

        self.setTestLog()

        self.setTestStatus()

        

    

        

    # Use regular expression split and capture 

    # groups to parse the log file for relevant 

    # test result data

    # ------------------------------------------------    

    def setTestOutput(self):

        file = open(self.filepath)

        

        regexString =  "-----------------TESTRUNNEROUTPUTBEGINS----------------"

        regexString += "([\w\s!-~]+?)"

        regexString += "-----------------TESTRUNNEROUTPUTENDS----------------"

        

        myArr = re.split(regexString ,file.read(), re.M )

        

        if ( len(myArr) > 2 ):

            lastResultIndex = len(myArr)-2

            self.testOutput = myArr[lastResultIndex]    

        else:

            self.testOutput = "No test detail found"

            

            

            

    # Use regular expression split and capture groups 

    # to parse test result data further, to determine 

    # specifically status --if test passed or failed.    

    # ------------------------------------------------

    def setTestStatus(self):

        regexString = "Test Suite success: ([\w]+)"

        myArr = re.split(regexString ,self.testOutput )

        if len(myArr)==3:

            self.testStatus = myArr[1]

            print "STATUS "+myArr[1]

        else:

            self.testStatus = "null"

    # Use regular expression split to parse test 

    # result data further, to grab formatted XML  

    # results to be parsed by Cruise Control.    

    # ------------------------------------------------

    def setTestLog(self):

        regexString = "<testsuites>[\w\s!-~]+?</testsuites>"

        myResult = re.search(regexString, self.testOutput, re.MULTILINE)

        self.testLogOutput = myResult.group()



        

    

    # Print the test output

    # ------------------------------------------------

    def printTestOutput(self):

        print self.testOutput

    # Print the test status    

    # ------------------------------------------------

    def printTestStatus(self):

        print self.testStatus





    # Write the test log output to the log file

    # ------------------------------------------------

    def writeTestLogOutput(self, filePath):

        writeFile=open(filePath, 'w')

        writeFile.write("<?xml version='1.0' encoding='UTF-8'?>\n"+self.testLogOutput)



    # Write the test status    

    # ------------------------------------------------

    def writeTestStatus(self, filePath):

        writeFile=open(filePath, 'w')

        print("WRITING -- "+self.testStatus)

        writeFile.write(self.testStatus)

        



# Entry point

try:

    # Get the filename from the command line arguments 

    

    logPath = sys.argv[1]

    testStatusPath = sys.argv[2]

    testLogOutputPath = sys.argv[3]

    

    classInstance = LogParser(logPath)

    classInstance.writeTestStatus(testStatusPath)

    classInstance.writeTestLogOutput(testLogOutputPath)

except EOFError:

    pass

Save this file as flashLogParser.py and save it in a folder called python in your project. You’ll need python up and running on your machine. Instructions for getting it up and running are here

Amend Build script to build main file *if* tests pass

Now we have our result as a simple text file with just the word true or false in it, we can load this into ANT and use it to determine if our build has succeeded or not. The ANT target looks something like this:

  <target name="parseFlashLog" depends="runTestHarness">

   <py-run script="python\flashLogParser.py" optimize="0">

     <arg value="${flashlog.location}"/>

     <arg value="${flashStatus.location}"/>

     <arg value="${flashOutput.location}"/>

   </py-run>   

 </target>

 <target name="readTestStatus" depends="parseFlashLog">

   <loadfile property="testStatus" srcFile="${flashStatus.location}"/>

 </target>

  <target name="determineTestHarnessSuccess" depends="readTestStatus">

   <fail>

     <condition>

       <not>

         <equals arg1="true" arg2="${testStatus}" />

       </not>

     </condition>

   </fail>

 </target>

Add the following properties to your build.properties file:

# ------------------------

# file names

# ------------------------

main.mxml.source=main

testHarness.name=testHarness

main.swf.name=savings




# ------------------------

# paths

# ------------------------

lib.dir=lib 

outputdir=bin

mxmlc.dir = C:/Program Files/Macromedia/Flex/bin 

flash.debugPlayer=${lib.dir}/SAFlashPlayer.exe

src.path = src

asunit.dir = #PATH_TO_ASUNIT 

testBuildDir = testFiles

mainBuildDir = main

flashlog.location=C:/logs/flashlog.txt

flashStatus.location=${basedir}/log/testStatus.txt

flashOutput.location=${basedir}/log/test-results/TEST-testOutput.xml


# -----------------------------------------------------------------------------

# all jar needed

# -----------------------------------------------------------------------------

svnjavahl.jar=${lib.dir}/svnjavahl.jar

svnant.jar=${lib.dir}/svnant.jar

svnClientAdapter.jar=${lib.dir}/svnClientAdapter.jar

jakarta-regexp.jar=${lib.dir}/jakarta-regexp-1.3.jar

commons-lang.jar=${lib.dir}/commons-lang-2.0.jar

jython.jar=${lib.dir}/jython.jar

pyAntTasks.jar=${lib.dir}/pyAntTasks.jar

Add the following to the beginning of your build.xml file:

  <path id="project.classpath">

   <pathelement location="${svnjavahl.jar}" /> 

   <pathelement location="${svnant.jar}" />

   <pathelement location="${svnClientAdapter.jar}" />

   <pathelement location="${jakarta-regexp.jar}" /> 

   <pathelement location="${commons-lang.jar}" /> 

   <pathelement location="${pyAntTasks.jar}" />

   <pathelement location="${jython.jar}" />  

 </path>  

 

 <taskdef resource="svntask.properties" classpathref="project.classpath"/>

 <taskdef resource="pyAntTasks.properties" classpathref="project.classpath"/>

Finally, change your ‘all’ target to look like this:

  <target name="all" depends="determineTestHarnessSuccess,compileMain"/>

Now, when we run our ANT build, it will only succeed if our test suite succeeds – another key piece in the jigsaw puzzle. You will also notice that we are now adding several java files to our library. These can be found here, here, here, here, and here. download them and put them in your lib directory. You will notice we have included some svn jars as well. The need for these will become clear in a step or two…

Try it out...

Give this a shot by running the 'all' target of your ant build. What you should find is that in your log folder there is a file called testStatus.txt and in the log/test-results folder there is a file called TEST-testOutput.xml if you open up the status file, you will see either 'true' or 'false' depending on the results of the test suite, and the XML file will have more detailed information on the test as a whole...

The next installment will tell you how to get cruiseControl up and running so the entire team can track your builds...

Posted by Paul BH at 01:54 PM | Permalink | Comments (0)
Tags | Flex | Continuous Integration | Best Practices | Test Driven Development

So, if you have got this far, your ant build will compile the test harness, which quits after a specified period of time, and then compiles the main build. So far, so good, but it’s not really rocking yet; what we now need to do is get our ANT build to know if our unit tests have actually passed or not (and later down the line by the build monitor). The trick here is to get your swf to somehow output to a textfile that can then be read & understood by the ant build.

As Luke had already blogged, the easiest way to do this is to take advantage of the under-publicized fact that you can get trace output to write to a log file when using the Flash debug player. What you need to do is amend your mm.cfg file (which is found at the root of your user directory) to look something like this:

ProfilingOutputFileEnable=0
ProfilingOutputDirectory=C:\Program Files\Macromedia\Flex\jrun4\servers\default\profiler\WEB-INF\ProfilerData
FrameProfilingEnable=0
ProfileFunctionEnable=1 
ErrorReportingEnable=1
TraceOutputFileEnable=1
TraceOutputFileName= C:/logs/flashlog.txt

For our concerns, the most important two lines are the last two. We will use this outputted log to our advantage very soon… The next step is to get ASUnit to trace its output as well as send it to the ASUnit UI. I built a simple class called TraceOutput which is very similar to the class that prints results to the UI. the main difference is that the ASUnit UI refreshes its output at the completion of every unit test, whereas we really only want the output traced once. The TraceOutput class is given test results at the same time as the testRunner sends results to the UI. This was done by adding the following to the testRunner class:

/**
 * @author Paul BH -- paul@eyefodder.com
 */
class com.asunit.util.TraceOutput {

	private var items : Array;

	private var version:String = "2.6.4";

	private var successes:Number = 0;

	private var failures:Number = 0;

	

		

	private static var instance : TraceOutput;

	

	public static function getInstance() : TraceOutput {

		if (instance == null)

			instance = new TraceOutput();

		return instance;

	}

	

	private function TraceOutput() {

		

	}

	public function printReport():Void{

		refreshOutput();	

	}

	public function refreshOutput():Void {

		trace(getFirstLine());

		trace(getTestResultString());

		trace(getSuccessString((failures == 0)));

printReportAsXML()

		trace(getLastLine());

	}

	

	public function addTest(item:Object, length:Number):Void {

		if(length == 1) {

			items = new Array();

		}

		items.push(item);	

	}

		

	private function getSuccessString(isSuccess:Boolean):String{

		var str:String = "Test Suite success: "+isSuccess;

		str += "\n";

		return str;

	}

	private function getVersionNumber():String {

		return version;

	}

	private function getFirstLine():String {

		var str:String = "-----------------TESTRUNNEROUTPUTBEGINS----------------";

		str += "\nVersion : " + getVersionNumber() + "\n";

		return str;

	}

	private function getLastLine():String{

		var str:String = "-----------------TESTRUNNEROUTPUTENDS----------------";

		return str;	

	}

	

	public function getTestResultString():String {

		successes = 0;

		failures = 0;

		 var str:String = "";

		 var fStr:String = "";

		 for(var i:Number=0; i<items.length; i++) {

		 	str += items[i].output;

		 	if(items[i].success) {

		 		successes++;

		 	} else {

		 		failures++;

		 		fStr += renderItemFailure(items[i]);

		 	}

		 }

		 return str + "\n" + getTestNumberOutput() + "\n" + fStr;

	}

	public function renderItemFailure(item:Object):String {

		var str:String = "-----------------\nItem Failed at :\n";

		str += "assertion : " + item.assertion + "\n";

		str += "message : " + item.message + "\n";

		str += "methodName : " + item.methodName + "\n";

		str += "className : " + item.className + "\n";

		return str;

	}

	public function getTestNumberOutput():String {

		return successes + " out of " + items.length + " Asserts passed.\n";

	}

public function printReportAsXML():Void{

		var str:String = getReportAsXML();

		trace (str);

			

	}

	private function getReportAsXML():String{

		var ind:String = "\t";

		var nl:String = "\n";

		var str:String = "<testsuites>"+nl;

		var suiteObj:Object = getItemsBySuite();

		for (var i in suiteObj){

			str += getSuiteAsXML(suiteObj[i],i);

		}

		str += "</testsuites>";

		return str;	

	}

	private function getSuiteAsXML(arr:Array, suiteName:String):String{

		var outStr = "";

		var ind:String = "\t";

		var nl:String = "\n";

		

		var max:Number = arr.length;

		var time:Number = 0;

		var tests:Number = max;

		var failures:Number = 0;

		var item:Object;

		for (var j:Number=0;j<max;++j){

			item = arr[j];

			time += item.executionTime;

			failures += item.success ? 0 : 1;

			outStr += ind+ind+getTestCaseAsXML(item)+nl;

		}

		var openingTag:String = "<testsuite name=\""+suiteName+"\"";

		openingTag += " failures=\""+failures+"\"";

		openingTag += " tests=\""+tests+"\"";

		openingTag += " time=\""+time+"\"";

		openingTag += ">";

		outStr = ind+openingTag+nl+outStr;

		outStr += ind+"</testsuite>"+nl;

		return outStr;	

	}

	private function getItemsBySuite():Object{

		var outObj:Object = {};

		var arr:Array = items;

		var max:Number = arr.length;

		for (var i:Number=0;i<max;++i){

			var ref:String = arr[i].className;

			if(outObj[ref]==undefined){

				outObj[ref] = [];	

			}	

			outObj[ref].push(arr[i]);

		}		

		return outObj;	

	}

	private function getTestCaseAsXML(inObj):String{

		var str:String = "<testcase name=\""+inObj.methodName+"\" time=\""+inObj.executionTime+"\">";

		if (inObj.success==false){

			str += "<failure>"+inObj.message+"</failure>";	

		}

		str += "</testcase>";

		return str;	

	}

}

Now, in order to get the asunit framework to actually add results to the TraceOutput class, we need to add one line to the renderTest method of com.asunit.framework.TestRunner :

public function renderTest(item:Object):Void {

	var lc:LocalConnClient = getLocalConn();

	lc["addTest"](item, length);

	TraceOutput.getInstance().addTest(item,length);

}

Next we want the class to print its results in the trace panel, so add the following line of code to your test harness file:

private function onTestsComplete(){

	TraceOutput.printTestResults();

	fscommand(“quit”);

}

Now, when you run your test output, you will see test results in your output panel (we use flashlog in Eclipse). something like this:

-----------------TESTRUNNEROUTPUTBEGINS----------------

Version : FMR 3.1.1

.F

1 out of 2 Asserts passed.

-----------------

Item Failed:

assertion : assertTrue

message : failingtest

methodName : test

className : com.TestClassTest

Test Suite success: false

<testsuites>

	<testsuite name="com.TestClassTest" failures="1" tests="2" time="0.008">

		<testcase name="testInstantiated" time="0.008"></testcase>

		<testcase name="test" time="0"><failure>failingtest</failure></testcase>

	</testsuite>

</testsuites>

-----------------TESTRUNNEROUTPUTENDS----------------

As you can see, there are two parts to the output. The first is the nice, human readable output, modeled very closely on the output in the ASUnit UI. The only real change to it is that instead of the red bar/ green bar for test suite result, I changed this to say Test suite success: true | false. This will prove useful in the next step when ANT needs to determine test suite success / failure. The second half is the test output printed out as a chunk of XML. This will get used by our build tool later down the line... In my next post, we will delve a little into python land to get the flash log parsed into a usable state for our build tool....

Posted by Paul BH at 01:42 PM | Permalink | Comments (0)
Tags | Flex | Continuous Integration | Best Practices | Test Driven Development

In the next step to CI Nirvana, I'm going to be looking at using ANT to build both our testHarness and the main mxml file. As I mentioned in my previous post, the job we need ANT to do is get it to compile the testHarness, wait for it to close, then compile your main build.

You can use ANT to compile flex using the mxmlc compiler. Create a build.xml file at the top level of your project and paste the following in there:

<?xml version="1.0"?>

<project name="myProject" basedir="." default="all">

  <property file="build.properties" /> 

  

  <target name="all" depends="runTestHarness,compileMain"/>



 <target name="properties">

   <fail unless="mxmlc.dir">The "mxmlc.dir" property must be set in

     ${basedir}/build.properties.</fail>

   <fail unless="flash.debugPlayer">The "flash.player" property must be set in

     ${basedir}/build.properties.</fail>

   <fail unless="main.mxml.source">The "savings.mxml.source" property must be set in

     ${basedir}/build.properties.</fail>

 </target>

<target name="compileUnitTests" depends="properties">

   <exec executable="${mxmlc.dir}/mxmlc.exe" dir="${mxmlc.dir}">

     <arg line="-aspath '${basedir}/${src.path}';'${asunit.dir}'"/>

     <arg line="-o '${basedir}\${outputdir}\${testBuildDir}\${testHarness.name}.swf'"/>

     <arg line="-configuration '${basedir}/flex-config.xml'"/>

     <arg line="'${basedir}/${testHarness.name}.mxml'"/>			

   </exec>

 </target>

  <target name="runTestHarness" depends="compileUnitTests">

   <exec executable="${basedir}\${flash.debugPlayer}" spawn="no" >

     <arg line="'${basedir}\${outputdir}\${testBuildDir}\${testHarness.name}.swf'"/>

   </exec>

 </target>

  <target name="compileMain">

   <exec executable="${mxmlc.dir}/mxmlc.exe" dir="${mxmlc.dir}">

     <arg line="-aspath '${basedir}/${src.path}';'${asunit.dir}'"/>

     <arg line="-configuration '${basedir}/flex-config.xml'"/>

     <arg line="-o '${basedir}/${outputdir}/${buildDir}/${main.swf.name}.swf'"/>

     <arg line="'${basedir}/${main.mxml.source}.mxml'"/>			

   </exec>

 </target>

Your build properties file should look something like this:

# ------------------------

# file names

# ------------------------

main.mxml.source=main

testHarness.name=testHarness

main.swf.name=main




# ------------------------

# paths

# ------------------------

lib.dir=lib 

outputdir=bin

mxmlc.dir = C:/Program Files/Macromedia/Flex/bin 

flash.debugPlayer=${lib.dir}/SAFlashPlayer.exe

src.path = src

asunit.dir = #PATH_TO_ASUNIT 

testBuildDir = testFiles

mainBuildDir = main

A few notes on our build script:

• We put a copy of the flash debug player in the lib folder of our project. This means that when we move the project to another machine (or running on the server…) we don’t need to worry about finding the absolute path to our debug player.


• Our project outputs go in the bin directory. Test output goes in a subdirectory called testFiles, and main output goes in one called, erm ‘main’ . The reason we built it this way is that when developing our webservices are not always ready when we need them, so our main app is set up to run off mock objects, which are also shared by the test harness. These are in a folder in bin cryptically called mockObjects. Later down the process, it’ll become clear why we have all this stuff in the bin directory…


• As you can see, we specify the path to the asunit directory, as this is not part of our current project. If you have other classpaths you want to add, you can just add them to the –aspath argument of the compiles.

So, if you now run your ant build, you should see your test harness run , then close after a while, and then your main build compiles. So far, so good, but it’s not really rocking yet; what we now need to do is get our ANT build to know if our unit tests have actually passed or not. In order to do this, we will need to get ANT to understand the result of the unit tests, and this is what we will look at next...

Posted by Paul BH at 01:29 PM | Permalink | Comments (0)
Tags | Flex | Continuous Integration | Best Practices | Test Driven Development

The first step for getting your CI build up and running will be to get all your basic pieces in place on your local machine.
Some of this stuff (I hope) you will already have in place, but for the sake of being complete I'll include it here...

1. Set up a source control repository.

Regardless of whether you do CI or not, you really should be putting your work under source control. We used Subversion because its easy to use, is free, and there are great integration plugins for windows explorer, mac OSX and eclipse. You can get information on installing subversion here

2. Install ASUnit.

So, our next step was to start using a unit testing framework. We settled on ASUnit, primarily because most of the team had already used it in other projects. I was pretty comfortable with the source, having digged into it a few times and furthermore, Luke had blogged about how one might go about CI with asunit already. We will be playing around with the source of ASUnit later, but for now, lets check everything works as planned…

3. Create a simple Class and a test suite for it

As you create your classes, you should be creating your unit tests in parallel. Our testHarness.mxml is the entry point to our test suite, and is a pretty simple affair. All it does is instantiate our AllTests class, which basically steps through your classpath, adding tests as it finds them. ASUnit ships with a jsfl plugin to do automatically create these using the Flash IDE. As we were using Eclipse / FDT as our development platform, this wasn’t a huge amount of use to us, so Johannes rewrote the original jsfl files to run as a python script. He’s blogged about it here . Go ahead and make a couple of classes get these scripts to make the bare bones unit tests for you – its really rather clever…

4. Create a testHarness.mxml file to run your tests

So, now you have your test class created, its time to take a step back and think about how everything is going to fit together. Basically our work flow is along these lines:

1. Developer does some work.
2. Developer runs test suite locally.
3. Developer compiles build locally.
4. Developer commits code changes to the repository.
5. Build monitor checks periodically for changes to the repository. When there has been a change it:
6. Attempts to compile & run test suite. If this succeeds it:
7. Attempts to compile main build.
8. Build monitor then notifies either entire project team on successful build (hooray!) or the development team on unsuccessful build (boo).

Other than getting the developer to do some work, the tricky part of the process is actually number 6. In order for this to work, we need to do the following:

1. get our test harness to quit when all tests have completed so that our ant build can carry on and try to compile the application.
2. Get our ant build to somehow understand if the unit tests have completed successfully or not.

There are two approaches to getting the first task complete. The simplest is to get your test harness to quit after what you consider to be a safe period of time, say 30 seconds or so. In order to do this, you could just do something like this:

private function initApp():Void{
	var myTests:AllTests = new AllTests();
	setInterval(this,”onTestsComplete”,30000);
}
private function onTestsComplete(){
	fscommand(“quit”);
}

This is not ideal however, as you have to keep an eye on the length your unit tests are taking to execute – if it gets to be longer than your specified safe period, you will need to adjust the timeout period. Peter Hall ended up reworking ASUnit to an event based model such that our allTests class triggers a testComplete event when all the tests have finished running. In order to do this, he significantly changed the way that ASUnit works, so if you are looking for a simple solution, go for the option I have put in here; it’ll get you up and running…

5. Create an index.mxml file that will be used for the main build.

Now, you also need to have yourself an entry point for the application itself. So, go ahead and make one – it doesn’t matter too much right now what it is – start simple and work up. On a side note, I’d suggest you don’t launch in and try to retro-fit a suite of unit tests to a pre-existing app / codebase, unless you have a fair chunk of time on your hands, or are about to take on a hefty rework or refactor – it probably isn’t worth it…

OK - thats the end of part one. It may not seem like you've got very far, but you have the almost all the main elements in place for your CI suite. The next step will look at tying the pieces together by creating an ANT build that will be used to run your unit tests and compile your app

Posted by Paul BH at 01:53 PM | Permalink | Comments (1)
Tags | Flex | Continuous Integration | Best Practices | Test Driven Development

The contract I am currently working on is for a large financial institution in Boston. They want to explore new ways of working in our small business unit with the hope that some of our work may make it into the mainstream of the company.

We are not only experimenting with the projects we work on, but also trying out new (for the company at least) ways of working.

So, when my manager came to me late last year and started asking what we could look at next, there was a challenge that I and my workmates (Peter Hall, Ikezi Kamanu and Johannes Nel) had been looking to get our teeth into for a while — Continuous Integration with Flex.

So, here’s the trouble: I’m a lazy person, and I need to see tangible results for any effort I put into a task. Then I started looking into Continuous Integration, and finally I could see I would get a real and immediate benefit from unit testing. Have a look at Martin Fowler’s excellent article on CI for full details. This will tell you all the reasons from a technical standpoint why CI is a great idea. The one that stuck in my mind was the idea that whenever code is changed, a build is run, and if it passes tests placed against it, the project team gets to know about it. This isn’t just the developers, but the entire project team.

That for me is the real beauty of this setup, no more project managers, designers, business analysts or QA guys tapping me on the shoulder asking to see where we are. Because they are always kept up to date with where we’re at, there’s more visibility in the development process and less mysticism and panic.

So, having decided to try and get a CI process running , we set about getting this working with Flex. Most of the pieces are in place for this; the work we had to do was in joining the dots. I’ve put this together as a step by step process, although for many of them I’ll direct you to where someone else has explained it better than I ever could.

Posted by Paul BH at 01:39 PM | Permalink | Comments (1)
Tags | Flex | Continuous Integration | Best Practices | Test Driven Development | ASUnit