Every test design technique in this course belongs to one of two families, depending on a single question: do you look inside the program or not? This lesson introduces both families, shows why neither can test everything, and explains the order the professor insists on.
Imagine you are asked to test a vending machine. There are two ways to go about it:
These are exactly the two testing strategies.
No. Each family hits its own wall, and the two walls have names.
To be certain from the outside, you would have to try every possible input. This is exhaustive input testing, and it is impossible: remember the calculator in 2.1, where two three-digit numbers and four operators already gave 4,000,000 tests.
It is even worse than that, because the order of inputs matters too. Think of the power button on a TV remote. Pressing it turns the TV on or off, depending on what happened before. So it is not enough to try every input once; you would have to try every sequence of inputs, and the number of sequences has no end.
To be certain from the inside, you would have to follow every path through the code. This is exhaustive path testing, and it is also impossible, because loops repeat paths again and again: a loop that can run 1, 2, 3, … times creates a new path for every number of repetitions.
And even if you could test every path, you would still not be done. Every path could work exactly as the code says, and the user could still be unhappy because the code does the wrong thing, or misses something the user needed. That is bug definition 5 from 1.6. Path testing only checks the code that exists; it cannot find code that should exist but does not.
| Black box | White box | |
|---|---|---|
| You see | Only the interface (input and output) | The code and its paths |
| Other names | Input-output driven, data-driven | Logic-driven, transparent box |
| Tests come from | The specification | The program’s logic |
| The impossible ideal | Exhaustive input testing | Exhaustive path testing |
| Why it is impossible | Too many inputs, and input order matters | Loops multiply the paths; even all paths can leave bug definition 5 |
Black box first. The reason is bias.
When you read the code, you start to think like the programmer, and the programmer believes the program works. You see how the code handles a case, and you unconsciously test that it does what it does, instead of testing what it should do. That bias leaks into your tests.
So the professor’s order is:
Unit testing can be either one. If you test a method only through its inputs and outputs, it is black box. If you read its code and design tests for its branches, it is white box. The level of testing (unit, integration, system) and the strategy (black or white box) are separate choices.
Exam radar: Order: black box first, then white box. Reason: bias. Knowing the code makes you think like the developer.
Black box is limited by exhaustive input testing. White box is limited by exhaustive path testing.
Common mistake: writing “white box is better because it sees more”. Seeing the code is exactly what brings the developer’s bias, and white box testing cannot find missing features at all. The two strategies complement each other; neither replaces the other.
Why do we usually do black box testing before white box testing?
Because of bias. Seeing the code makes the tester think like the developer, and the developer is biased toward believing the program works. A tester who has read the code tends to test what the code does rather than what it should do.
So we start with black box tests while the mind is still free, and only when we run out of ideas do we open the box and add white box tests.
Name the impossible ideal that limits each strategy, and give one reason it is impossible.
A team claims: “We tested every path through our code, so the program has no bugs left.” Explain why this claim is wrong even if every path really was tested.
Path testing checks only the code that exists. Every path can behave exactly as written and the program can still be wrong:
In both cases the user is still unhappy, which is bug definition 5 (1.6). So even complete path testing cannot show the program has no bugs; it only shows the existing paths behave as coded.
For each test idea, say whether it is black box or white box:
else branch that no test reaches, then write a test for it.sqrt(16) and check the result is 4, without opening the method.while loop runs zero times, once, and many times.Is unit testing black box or white box testing? Explain.
It can be either. Unit testing is a level of testing (one small unit, such as a method or class), while black box and white box are strategies.
A media player has one Play/Pause button. A tester checks that pressing it once starts the music, and concludes the button is fully tested. Using the idea of exhaustive input testing, explain what is missing.
The button’s effect depends on what happened before, exactly like the TV remote’s power button: when music is playing it should pause, and when it is paused it should play. So the input “press the button” does not have one expected result; the expected result depends on the earlier sequence of inputs.
A fuller test set must try the button in different states, at least: pressed while stopped (→ play), pressed while playing (→ pause), pressed while paused (→ play again). This is why exhaustive input testing is even more impossible than counting inputs suggests: the order of inputs matters too.