White box testing is done; Lecture 5 turns to the black box. Before the new techniques, the professor started with people: how a tester should work with the developers whose code they are breaking, why bringing in an outside testing company is not free, and how the two “infinite” problems of testing line up with the tools of this course.
The developer wrote the code and expects it to work. That expectation is a bias, and it is the reason testing is a separate role. The tester’s job is the opposite of the developer’s: break the software to show that bugs exist.
But the professor was careful about how a tester says it. A tester should never be the team’s “nightmare” or “bad news”. Think of two ways of telling a friend there is spinach in their teeth: laughing at them in front of everyone, or quietly saying “hey, quick thing, let me help”. The information is the same; only one of them keeps the friendship.
| Not like this | Like this |
|---|---|
| “Your code is terrible!” | “I found a bug. Can we look at it together?” |
| The developer gets defensive and starts to resent testers. | “Thanks! Let’s fix it.” The team stays a team. |
Exam radar: the tester’s main goal is to show that bugs exist, not to fix them. But the tester is also responsible for following up, so that as many of the reported bugs as possible really get fixed.
The professor told the class about the US Department of Defense and its research agency DARPA. For safety-critical software, they want a third party to test it: an independent company, not the development team’s own QA group. An outsider has no reason to protect the code, so the bias problem from the previous section disappears.
Then the professor asked the class: what is the most obvious problem with third-party testing? Several students wrote long, careful answers. The professor said they were “overthinking”. The answer wanted was one word.
How to write the answer: cost. An outside company must learn the software from zero and bring its own people, so the cost of testing goes up a lot. When a question asks for “the most obvious” problem, give the obvious one first and explain it in a sentence or two.
A student added a smart point, and the professor agreed: cost is relative. If a failure can kill people (weapons, aircraft, medical devices), the cost of failure is already enormous, so the extra testing cost looks small next to it. That is exactly why safety-critical projects pay for independent testing anyway.
The professor added an observation from industry. Companies love penetration tests (paying security experts to attack the system), because a pen test produces a report you can put on your boss’s desk. But training developers to write safe code, plus reviews (2.6), is often the better investment in the long run. It is just harder to sell, because its benefit arrives later and quietly: bugs that are never written leave no report behind.
The professor then connected everything learned so far. Each side of testing has its own “infinite” problem, and each has its own answer:
| White box (Lectures 3–4) | Black box (Lectures 5–6) | |
|---|---|---|
| The infinite problem | Too many paths through the code to cover them all | Too many inputs and outputs (and orders of them) to try them all |
| The answer | MC/DC, the optimal coverage criterion | Equivalence partitioning + boundary value analysis (this lecture), then decision tables and cause-effect graphs (Lecture 6) |
In both cases the job is the same: shrink an infinite number into a small, finite number of tests that still gives you confidence. On the white box side you already did this with MC/DC; on the black box side, the next lesson (5.2) introduces the first tool.
Diagram loads as you scroll.
Common mistake: writing that the tester’s job is to fix the bugs they find, or that a bug report is the end of the tester’s work. The tester reports, discusses, agrees a priority, and follows up; the developer fixes.
What is the most obvious shortcoming of third-party testing?
Cost. An independent company must learn the software from zero and bring its own people, so testing becomes much more expensive than using the team’s own QA.
(The cost is relative: for safety-critical software, where a failure can kill people, the cost of failure is so large that the extra testing cost is worth paying.)
A tester finds a bug. Describe what should happen from the bug report until the bug is closed, and say who does what.
So the tester shows that the bug exists and follows it up; the developer fixes it.
A developer argues: “Testers only slow us down. If they are so clever, they should fix the bugs themselves.” Discuss.
The developer is mixing up two different roles. The tester’s main goal is to show that bugs exist. The developer wrote the code and expects it to work, which is a natural bias; the tester’s job is deliberately the opposite, to break the software. If testers also fixed the bugs, they would take on the developer’s ownership of the code, and with it the same bias, and nobody would be left whose job is to find what is wrong.
That does not mean the tester’s work stops at the report. A good tester reports bugs in a way that keeps the team a team, discusses disagreements and priority (high, medium, low) with the developer, and follows up until the bug is fixed. So the tester does not slow the team down: the tester finds the bugs and makes sure they get fixed; the developer fixes them.
A company has budget for one of two things: a yearly penetration test, or a programme that trains its developers in secure coding plus regular code reviews. Which did the professor suggest is often better in the long run, and why do companies still tend to choose the other?
Training plus reviews is often better in the long run. It prevents security bugs from being written at all and catches the remaining ones early, when they are cheap (the ×10 rule of 1.3).
Companies still tend to buy the penetration test because it gives a visible result, a report you can show to your boss. The benefit of training arrives later and is invisible (bugs that are never written leave no trace), so it is harder to sell, even though it is often the better choice.
Both white box and black box testing face an “infinite” problem. Name each problem and the technique the course uses to answer it.
In both cases the goal is the same: a small, finite number of tests that still gives confidence.