How to make acceptance testing agile

October 25, 2007 by Vaibhav

I wrote about Acceptance Testing earlier and why it is a very important tool for agile development. In this post, I will talk about how we can make acceptance testing really agile. Actually, it's nothing more special than making Acceptance Testing a part of your Continuous Integration (CI) (an XP Primary Practice). Continuous Integration is the practice of merging, building, and testing code on a continuous basis (almost instantaneous basis). Cruise Control is one of the most popular frameworks which allows you to implement CI.

Solution

Ok, so back to the topic. How do you integrate acceptance testing into a Continuous Integration cycle? Of course, to do so, you need to automate the acceptance testing. There are many frameworks available on the Internet for automating acceptance testing. Some are specific to a technology, while others are more general. The one I want to recommend is Selenium. Using Selenium, you can acceptance test your 'web-based' applications against a variety of platforms and browsers. Follow the above link to read more about Selenium. It is an open-source software and relatively simple to use.

Considerations

There are some points that you need to consider while employing Selenium or any other acceptance testing framework. And the choices you make depend on your situation. You can integrate Selenium into your CI cycle. If you use Cruise Control, then you can write scripts that automatically call and run Selenium test cases for your application with every build. So, here are the things to consider:

• If you make the acceptance testing as part of your continuous build cycle, is it causing your agility to suffer because suddenly every build cycle is taking much longer? This can happen if you have a large number of test cases. This may not be noticeable in the beginning but as your code base grows, you can run into very slow build times.
• One way to do it is to run the acceptance tests manually. But this may not work because your team will not get acceptance testing feedback with a lot of regularity; this defeats the purpose because ideally you want quick feedback so that you can correct the error before moving on. This may also be the case if you schedule these tests using some external scheduler because then you will not get a feedback upon every build.
• Another solution is to trigger an independent running of these test cases whenever there is a build cycle initiated during the CI process. This allows the build to complete quickly and still you will get acceptance test results for every build.
• A third solution is to only include certain tests in the CI cycle and run the full test case suite independent of the development cycle. This allows you to keep your important test cases within the CI cycle.

Depending on your situation, you can choose any of the approaches above; or even create a new one of your own. How do you automate your Acceptance Testing? What approach do you use? Do you face the problem described above?

Permalink