Acceptance Testing: What, Why and How

What is acceptance testing?

Acceptance testing is a black-box testing performed on a software prior to its delivery. It involves running a suite of tests on a completed system (ref: Wikipedia). These test suites are made up of multiple tests or test cases. Each test case consists of a sequence of steps to perform which emulate the use case that is being tested; it also contains input data (if required) as well as the expected output. The result of a test case is either a pass or a fail.

WHY?

Although acceptance testing traditionally takes place at the end of development or major milestones, in agile software development acceptance testing needs to be performed at the user story level. There are several reasons for why this is important:

  • A passed test case becomes a measure of completeness of a user story; that is, a user story cannot be considered complete till it has passed all acceptance tests associated with it.
  • Even though there is thorough unit testing performed, this is not enough. Unit tests, by their nature, test for a localized used case and are not concerned about the overall system.
  • When we have iterations longer than a couple of weeks, it becomes easy to loose focus on initial agreements; acceptance test cases made for each story at the beginning of each iteration help the developers to keep things within the expectations.
  • Acceptance test cases can serve as an excellent guide to developers to better interpret the requirements from a user story

The Acceptance Test Case

An acceptance test case has the following primary objectives:

  • The Product Owner should be able to verify and validate that the user story has been implemented as it was supposed to.
  • The developers should be able to check if what they have developed is in accordance with the requirements.
  • We do no development that cannot be verified.

An acceptance test case is a human level test which determines whether the product meets the expectations of the end user.

HOW?

When creating acceptance test cases, the following points (some obvious) should be covered:

  • All scenarios of the User Story should be covered in the test cases, and the expected outcome mentioned so that a tester can verify each function and use case.
  • The acceptance test case needs to be at a broad overview level and not have details such as field validations, etc. because those will be covered by unit test cases.
  • The test cases should precisely define the steps and their sequence that the tester should take; the inputs; and the expected outputs.
  • When testing a non-UI based module, an acceptance test case can be an automated test case which also records the response. This will allow you to examine the output and see if it is acceptable.
  • The acceptance test cases need to be reviewed and approved by the Product Owner in order to ensure that the coverage is complete.

Once test cases are created and approved, they should be given to the developer prior to start of development work. This allows the developers to get better understanding of the requirements as compared to what they might otherwise understand from just the User Story. This implies that the test cases need to be made before the actual development starts.

There are some tools for running these kind of test cases automatically, but it is very common to have a group of testers run these manually. How do you manage acceptance testing? What tools and best practices do you use?

7 thoughts on “Acceptance Testing: What, Why and How”

  1. Interesting
    But when should testers write scenarios?
    If the answer is after sprint planning meeting, then developers should wait while tester write the test case and approve it?

  2. Actually, the test cases should be written as soon as we have user stories available. This can be an ongoing exercise as the Product Backlog is filled up.

    Even if you don’t do that, the developers can ‘begin’ their work and consult the test case part of the way into their development. It doesn’t take too long to write an acceptance test case for a typical story.

  3. “But when should testers write scenarios?”

    The answer is, to use a phrase from Lean Software Development, “at the last responsible moment”. Which in this case is as shortly before they are needed as possible – shortly before the iteration the features will get implemented.

    Writing them earlier would be wasteful, for at least two reasons: you are investing work into things that

    – don’t pay back soon, and
    – have a high probability of experiencing change before they even have a chance to provide value (they might not even be needed at all, because the feature might actually never get implemented)

    As an aside, tests shouldn’t be written by testers in isolation, but at least in close collaboration with the customer/product owner.

  4. Thank you so much for this useful information.

    I am new to the agile methods. Can I ask please if the acceptance tests can not be automated, when we will ensure that they are passed? Is it just before releasing or can be done earlier (in the development)? Please provide explanations as much as you can.

    Regards,
    Sultan

  5. Who has to write Acceptance Cases? Whether QA or BA or Client or PM? Because, just by seeing user stories, its very vague to write Acceptance test cases?

Leave a Reply

Your email address will not be published. Required fields are marked *