What is acceptance testing?
Acceptance testing is a black-box testing performed on a software prior to its delivery. It involves running a suite of tests on a completed system (ref: Wikipedia). These test suites are made up of multiple tests or test cases. Each test case consists of a sequence of steps to perform which emulate the use case that is being tested; it also contains input data (if required) as well as the expected output. The result of a test case is either a pass or a fail.
Although acceptance testing traditionally takes place at the end of development or major milestones, in agile software development acceptance testing needs to be performed at the user story level. There are several reasons for why this is important:
- A passed test case becomes a measure of completeness of a user story; that is, a user story cannot be considered complete till it has passed all acceptance tests associated with it.
- Even though there is thorough unit testing performed, this is not enough. Unit tests, by their nature, test for a localized used case and are not concerned about the overall system.
- When we have iterations longer than a couple of weeks, it becomes easy to loose focus on initial agreements; acceptance test cases made for each story at the beginning of each iteration help the developers to keep things within the expectations.
- Acceptance test cases can serve as an excellent guide to developers to better interpret the requirements from a user story
The Acceptance Test Case
An acceptance test case has the following primary objectives:
- The Product Owner should be able to verify and validate that the user story has been implemented as it was supposed to.
- The developers should be able to check if what they have developed is in accordance with the requirements.
- We do no development that cannot be verified.
An acceptance test case is a human level test which determines whether the product meets the expectations of the end user.
When creating acceptance test cases, the following points (some obvious) should be covered:
- All scenarios of the User Story should be covered in the test cases, and the expected outcome mentioned so that a tester can verify each function and use case.
- The acceptance test case needs to be at a broad overview level and not have details such as field validations, etc. because those will be covered by unit test cases.
- The test cases should precisely define the steps and their sequence that the tester should take; the inputs; and the expected outputs.
- When testing a non-UI based module, an acceptance test case can be an automated test case which also records the response. This will allow you to examine the output and see if it is acceptable.
- The acceptance test cases need to be reviewed and approved by the Product Owner in order to ensure that the coverage is complete.
Once test cases are created and approved, they should be given to the developer prior to start of development work. This allows the developers to get better understanding of the requirements as compared to what they might otherwise understand from just the User Story. This implies that the test cases need to be made before the actual development starts.
There are some tools for running these kind of test cases automatically, but it is very common to have a group of testers run these manually. How do you manage acceptance testing? What tools and best practices do you use?