UI Testing Framework + Continuous Integration?

Question

So I have an application that I inherited and I want to build up an automated test suite around it. The application wasn't designed with testability in mind, and the code is a "big ball of mud". My plan was to use a UI Automation testing framework and create a suite of tests at the UI level until I had enough coverage to allow me to start refactoring with confidence and introducing some seams into the code to improve testability and design.

It's a .Net WinForms application, and the two frameworks I'm aware of are:

NUnitForms

and

Project White

From what I've read both frameworks pose issues when trying to run as part of an automated build (Continuous Integration) due to the fact that most CI products run as a Windows Service and if the UI uses modal dialogs the application will die a horrible death. I'm using CruiseControl.Net as my CI tool.

Does anybody have any suggestions to work around this issue? An alternative framework to use that may make the situation better?

Thanks,

Dylan

Answer 1

NUnitForms has a "hidden desktop" feature that will let you run unit tests from cc.net.

http://automaticchainsaw.blogspot.com/2007/09/winforms-testing-using-nunitforms.html http://automaticchainsaw.blogspot.com/2007/09/hidden-desktops-and-nunitforms.html

Answer 2

You can actually run cruise control via the console app so it can have interactive desktop access. It won't recover automatically if the server is rebooted or it crashes, but at least you can do it.

That said, the approach most people take with automated UI testing (winforms, wpf or web) is to run all the non-interactive tests via the build server. Once those tests pass, then they deploy the application to a test environment and manually trigger a test run against the newly built version of the code.

This gives people the chance to reset the test environment (important for UI testing) as well as check that he new version of the application was built correctly and that all unit tests passed. After all, there's no point running the UI tests if you know the unit tests fell over. :-)

Answer 3

I didn't try it yet, but there is the UI Automation Framework from Microsoft:

Answer 4

We ran the continuous integration acceptance tests in console mode instead of as a Windows service within a logged in Virtual PC. That worked for us.

Answer 5

Take a look at this approach. There is detailed information in the project's wiki.