Theory: The Five Definitions of a Bug
This is the last piece of Lecture 1, and the professor leaned on it more than on anything else in the lecture. It ties together everything before it: the specification (1.1), verification and validation (1.2), and the gap between ideal and real software (1.3).
Patton’s Five Definitions
According to Patton, a software bug exists when at least one of five things is true. Picture a restaurant, where the menu plays the part of the specification:
| # | The definition | The restaurant picture | Can verification catch it? |
|---|
| 1 | The software does not do something the specification says it should. | A dish printed on the menu that the kitchen cannot make. | Yes. Compare the software with the specification. |
| 2 | The software does something the specification says it should not. | The menu promises a nut-free dish, and it arrives full of nuts. | Yes. Same comparison. |
| 3 | The software does something the specification does not mention. | A surprise side dish that is not on the menu at all. | Yes. The extra behaviour shows up against the specification. |
| 4 | The software does not do something the specification does not mention but should. | The menu forgot to list water, which every customer expects. | No. The specification itself is flawed. Only validation can catch it. |
| 5 | The software is difficult to understand, hard to use, slow, or the user sees it as just not right. | The right food, but it arrives two hours late and cold. | Mostly no. It is judged by user expectations, so it is subjective. |
Definitions 1 and 3 are the “missing” and “extra” parts of the two-circle picture in 1.3; definition 3 is exactly the dark-mode story told there.
The Famous Question: Can Verification Detect Bug 4?
The answer is no, and the reason is worth writing out in full.
Verification checks the software against the specification. But in Bug 4, the specification itself is what is wrong or incomplete. Checking the menu against the menu will always match, even if the menu forgot to list water. To notice that water is missing, you must step outside the specification and ask the customers what they actually need, and that is validation.
The professor added: when the specification is already faulty, whatever you build from it is a bug by default.
Exam radar: “Can verification detect Bug 4?” No. Verification uses the specification as its reference, and in Bug 4 the specification is the thing that is wrong. Only validation, which goes back to the user’s real needs, can reveal it.
Bug 5: Different in Kind
The first four definitions are about the software and its specification. The fifth is about the user’s expectations, so it is subjective: one user finds a screen clear, another finds it confusing. The professor called it the hardest to eliminate.
How to Write the Answer
How to write the answer: discussion answers come in two paragraphs. In the first, state the principle or foundation that is relevant. In the second, explain what it means for the question in front of you. Two smaller habits protect your marks:
- If the question asks why, begin with “This happens because…” and make the sentence after “because” your real answer.
- If the question is yes or no, write the word yes or no first. Do not make the marker guess.
Common mistake: if the question says find the bug, give the bug, not the fix. “The condition should be >=” is a fix; “an order of exactly $50 is charged shipping although the specification says it ships free” is the bug.
Worked Examples
Exercise 1
(Discussion) Discuss why testers should not aim to prove that software is correct.
Show solution ↓Hide solution ↑
Solution
The principle. ISTQB Principle 1 says that testing shows the presence of defects, not their absence. Because exhaustive testing is impossible, as Principle 2 states, no number of passing tests can ever prove that the software is correct. The real objective of testing is therefore to find bugs.
What it means here. A tester who sets out to prove correctness carries a mindset that shapes every test case, and will subconsciously choose inputs that make the software pass. This is the same bias that makes developers poor testers of their own code. A tester who sets out to break the software will write aggressive tests and expose more defects, which is what testing is for.
Exercise 2
Can verification detect a bug of type 4 in Patton’s list? Answer and justify.
Show solution ↓Hide solution ↑
Solution
No. This happens because verification checks the software against the specification, and in a type 4 bug the specification itself is incomplete: the software fails to do something the specification does not mention but should.
When the reference is faulty, comparing the software with it always “passes”, just as checking a restaurant menu against itself always matches even if the menu forgot to list water. The omission can only be found by stepping outside the specification and checking against the users’ real needs, which is validation. Whatever is built from a faulty specification is a bug by default.
Exercise 3
A developer adds an undocumented export feature because customers may find it useful. Is this a defect?
Show solution ↓Hide solution ↑
Solution
Yes. Under bug definition 3, software that does something the specification does not mention contains a defect. The developer’s good intention does not authorise the additional behaviour. The feature may create security, privacy, compatibility, maintenance or user-interface consequences that were never reviewed or tested.
If stakeholders approve the feature, the team should update the requirements and specification through change management and then test it normally. Until that happens, the implementation exceeds the authorised scope and should be reported as a defect.
Exercise 4
The SRS for an online shop says: “Orders must be paid by card. Card numbers must never be stored after payment.” For each observation, give the number of the bug definition it matches.
- (a) The checkout page has no card-payment option at all.
- (b) Card numbers are kept in the database after payment.
- (c) The shop also offers payment by cryptocurrency, which no document mentions.
- (d) There is no way to see the order total before paying; the SRS never mentions it, but every shopper expects it.
- (e) Checkout works correctly but takes 40 seconds, and testers find the form confusing.
Show solution ↓Hide solution ↑
Solution
- (a) Definition 1: the software does not do something the specification says it should (accept card payment).
- (b) Definition 2: it does something the specification says it should not (store card numbers).
- (c) Definition 3: it does something the specification does not mention (an extra payment method).
- (d) Definition 4: it does not do something the specification does not mention but should. Verification against the SRS would not catch this; validation would.
- (e) Definition 5: slow and hard to use, judged by user expectations.
Exercise 5
Why did the professor call Bug 5 the hardest to eliminate?
Show solution ↓Hide solution ↑
Solution
This happens because Bug 5 is judged by the user’s expectations, not by the specification, so it is subjective. Definitions 1 to 4 compare the software with the specification (or with what the specification should have said), which gives a clear yes-or-no check. “Hard to use”, “too slow” or “just not right” depends on who is using it: one user finds a screen clear while another finds it confusing, so there is no single reference to verify against and no point at which everyone agrees the bug is gone.
Exercise 6
The specification says: “Orders of $50 or more ship free; smaller orders pay $5 shipping.” The code is:
int shippingFee(int total) {
if (total > 50) {
return 0;
}
return 5;
}
Find the bug.
Show solution ↓Hide solution ↑
Solution
Trace the boundary value the specification names, $50:
shippingFee(50): the test 50 > 50 is false, so the method returns 5.- The specification says an order of $50 ships free, so the expected result is 0.
The bug: an order of exactly $50 is charged $5 shipping, although the specification says it must ship free. This is bug definition 1 (the software does not do something the specification says it should). As the question asks to find the bug, the answer is the wrong behaviour, not the corrected code. A check on other values: shippingFee(51) returns 0 and shippingFee(49) returns 5, both as specified ✓, so the fault is only at the boundary.
Exercise 7
(Discussion) Explain why a product can pass every verification activity and still contain bugs.
Show solution ↓Hide solution ↑
Solution
The principle. Verification checks a product against its specification. Patton’s five definitions show that not every bug is visible from there: definition 4 is something the specification does not mention but should, and definition 5 is about the user’s subjective expectations. On top of that, testing shows only the presence of defects, never their absence.
What it means here. This happens because the reference used by verification can itself be wrong or incomplete. If the specification forgot something users need, the product conforms perfectly and still fails them; only validation against the real needs reveals it. Slow or confusing behaviour that the specification never quantified also slips through. So passing verification proves conformance to the specification, not that the product is free of bugs.