Theory: Acceptance Testing & the Limits of TDD
TDD and JUnit make sure the code does what the developers’ tests say. But the developers’ tests are not the final judge: the customer is. This lesson covers acceptance testing, the common misconceptions about TDD, and the places where TDD on its own is not enough.
Acceptance Testing
A builder can check every wall with a spirit level and every pipe for leaks, and the house can pass every inspection. The family still has to walk through it and say “yes, this is the house we wanted” before they accept the keys.
Acceptance testing checks that the system satisfies the customer. The professor called it “tricky”: customers cannot always be forced to take part, they are busy, and they are not trained testers. So in practice developers often write the acceptance tests themselves, from the customer’s requirements, and the customer reviews them.
Two tools from the slides, for awareness only (know that they exist; you do not need to use them):
- Selenium: automatically clicks through a web page in a browser and checks the result, as a user would.
- FIT: customers write test cases as tables (in HTML or Excel), and FIT runs those tables against the code. This lets non-programmers express what they expect.
Passing unit tests does not mean the customer accepts the program. Unit tests check the code against the developers’ understanding; acceptance tests check the product against the customer’s needs.
Misconceptions About TDD
| Myth | Truth |
|---|
| TDD is more work. | Done right, it does not add work; it reduces it (fewer late bugs, safer changes). |
| Pair programming means no one is responsible. | Both people share the responsibility. |
| Customer involvement means the customer does all the testing. | Developers usually write the tests from the customer’s requirements. |
| Automated unit tests replace acceptance tests. | No. Passing unit tests does not mean the customer accepts the program. |
| Refactoring means changing everything. | No. It is small cleaning steps, with the tests kept green. |
| You must automate everything. | No. |
Where TDD Struggles
TDD is a strong habit, but it has blind spots:
- All unit tests pass, but the customer is still unhappy. That is still a bug: bug definition 5 (see 1.6), software that the end user finds hard to use, slow, or simply not right.
- Errors in the specification itself. The tests are written from the same wrong idea as the code, so code and tests agree with each other and both are wrong. Like a student who copies a wrong formula into both the homework and the answer key: everything “checks out”.
- Database schemas. The schema must be designed before you can write tests against it, so it cannot grow test-first in small steps.
- Very large teams, where keeping everyone’s tests coherent is hard.
- Safety-critical systems, which also need reviews and formal checks; tests alone cannot show the absence of dangerous defects.
- User-interface look and feel, which is hard to express as an automated pass/fail assertion.
How to write the answer when asked when to stop writing tests in TDD: use the professor’s phrase, write tests until fear turns into boredom. You write tests while you are still afraid the code might be wrong; when every new test you can think of feels certain to pass, you have written enough.
Worked Examples
Exercise 1
Why did the professor call acceptance testing “tricky”, and how is the problem usually handled?
Show solution ↓Hide solution ↑
Solution
Because acceptance testing depends on the customer, and customers cannot always be forced to take part: they are busy and are not trained testers.
The usual solution is that developers write the acceptance tests from the customer’s requirements, and the customer reviews or approves them. Tools such as FIT help by letting customers express test cases as simple tables.
Exercise 2
What do Selenium and FIT do? How deeply do you need to know them for this course?
Show solution ↓Hide solution ↑
Solution
- Selenium automatically drives a web browser: it clicks through a web page and checks the result, as a user would.
- FIT lets customers write test cases as tables (HTML or Excel) and runs those tables against the code.
The professor said they are for awareness only: know that they exist and what they are for; you do not need to use them.
Exercise 3
A team says: “All 2,000 of our automated unit tests pass, so we can skip acceptance testing.” Is this right?
Show solution ↓Hide solution ↑
Solution
No. Automated unit tests do not replace acceptance tests. Unit tests check that each unit behaves as the developers expect; acceptance testing checks that the whole system satisfies the customer. If the developers misunderstood a requirement, the code and its unit tests share the same mistake and all pass. The customer can still reject the product, and by bug definition 5 that dissatisfaction is a bug.
Exercise 4
Correct each misconception: (a) “TDD is more work.” (b) “Pair programming means no one is responsible.” (c) “Refactoring means changing everything.”
Show solution ↓Hide solution ↑
Solution
- (a) Done right, TDD does not add work; it reduces it: bugs are caught immediately while they are cheap, and the tests make later changes safe.
- (b) In pair programming both people share the responsibility for the code.
- (c) Refactoring is a series of small cleaning steps that do not change behaviour, with all tests kept green after each step.
Exercise 5
Name four situations where TDD struggles, and explain why for two of them.
Show solution ↓Hide solution ↑
Solution
TDD struggles with: errors in the specification, database schemas, very large teams, safety-critical systems, user-interface look and feel, and customers who are unhappy even though all unit tests pass.
- Errors in the specification: the tests are written from the same specification as the code. If the specification is wrong, code and tests agree with each other and are both wrong, so every test passes.
- User-interface look and feel: whether a screen is pleasant and easy to use is hard to express as an automated pass/fail assertion, so it cannot be driven by unit tests.