Theory: Testing vs. Debugging & the Five Debugging Strategies
You met the words testing and debugging in Lecture 1. In this lecture the professor drew the line between them sharply, corrected a very common way of describing it, and then pointed the class to five strategies for the hard half of debugging: finding where a bug really comes from.
Two Different Jobs
Your phone battery dies after three hours. You open a battery check and it says: “Something is using too much power.” That is testing: it shows that a problem exists. It does not tell you which app is to blame.
So you look at the apps one by one until you find the bad one. That is locating the cause. Then you remove or update the app. That is fixing. Locating and fixing together are called debugging.
Diagram loads as you scroll.
The professor’s correction: the word “find”
In class a student said: “Testing finds bugs, and debugging fixes them.” The professor said this is close, but the word find is a problem. To find something means to know where it is, and testing does not tell you that. A bug may appear in one place while its real cause sits somewhere else, earlier in the program. So the professor restated the two goals:
The goal of testing is to show that the software has bugs. The goal of debugging is to locate the cause of a bug and fix it. Testing does not find the bug in the sense of knowing its location; that is debugging’s job.
Who does what
Usually two different groups do these jobs. Testers do the testing and send a bug report to the developers. Developers own the code, so they do the debugging.
Which half of debugging is harder?
Locating the cause, by far. About 90–95% of debugging effort goes into finding the cause; only 5–10% goes into the fix. Locating needs a deep understanding of the code, and once you truly understand the cause, the fix is usually easy. With the phone: finding the bad app takes a long time; removing it takes one second.
Why Found Bugs Stay Unfixed
A surprising question from class: when testers report many bugs, will the developers fix all of them? No, for three reasons.
- Cost and time. A fix needs time and money. If a game must ship tomorrow, a small bug may be left in because there is no time.
- Low risk. If a bug is minor and rarely happens, fixing it may not be worth the cost (the impact grid from 2.2 again).
- New bugs. Changing code to fix one bug can create another. You have seen it after an app update: chat works better, but now the camera fails. That possibility is one more risk to weigh.
From the slides: the cost of debugging. Debugging can take up to 50% of a developer’s coding time. In the IBM Rochester study, a bug that slips from the coding phase into the testing phase costs about 9 times more to fix, and one that slips from testing to real customers about 13 times more. It is the same lesson as the economics of testing in 1.3: late bugs cost more.
Five Ways to Locate the Cause of a Bug
These five strategies come from the slides. The professor did not walk through them in class, but told the class to know them, for a modern reason: in future you may ask an AI to debug for you, and to give it good instructions you must know the names and the strengths and weaknesses of each strategy.
One example runs through all five: you have lost your phone somewhere at home. How do you find it?
1. Brute force
You search every room with no plan, opening every drawer. You will find the phone, eventually. In code, brute force means scattering print statements everywhere or reading a full memory dump. It is the most common strategy and the least efficient. People fall back on it when they have no idea about the cause.
2. Backtracking
You start at the place where you noticed the phone was missing and retrace your steps: bedroom, then kitchen, then living room. In code, you start at the wrong output and read the code backwards, line by line, until you reach the point where the value went wrong. It works very well for small programs.
3. Induction
You collect clues. The phone rang in the kitchen last night. The dog was in the kitchen. There is a wet spot on the table. A pattern appears and you form a theory: the dog moved the phone. Then you check the theory. Induction goes from specific clues to a general theory. Its steps: collect the data, look for patterns, form a theory, prove it.
4. Deduction
You list every possible place: bag, kitchen, car, bedroom. Then you use tests to cross places out. You call the phone and hear it ring inside the house, so it is not in the car: cross out the car. Continue until one place is left, then prove it. Deduction goes from a full list of possible causes down to the one real cause.
Diagram loads as you scroll.
5. Debugging by testing
You call the phone to hear where it rings. Each new test gives a new clue. In code, you write extra test cases whose purpose is to pinpoint exactly where the bug starts.
The five side by side
| Strategy | Lost-phone example | Strength | Weakness |
|---|
| Brute force | Search every room with no plan | Needs no idea about the cause | Very slow; drowns you in information |
| Backtracking | Start where you noticed it missing, go back step by step | Very good for small programs | Hard for big programs, where the path back is long |
| Induction | Collect clues, find a pattern, form a theory, check it | Fast when the clues show a pattern | The theory can be wrong if the clues are incomplete |
| Deduction | List all places, cross out the wrong ones | Careful and organised | Fails if the list of possible causes is incomplete |
| By testing | Call the phone and listen for the ring | Shows exactly where the bug starts | You must write extra test cases |
Exam radar: you may never apply these strategies by hand, but you must know the names and the strengths and weaknesses of each, so you can tell an AI what to do. A likely question: what is the difference between induction and deduction?
Rules for Fixing a Bug
Because a fix can create new bugs, fixing follows three rules:
- Run regression tests after the fix. Regression testing means running the old tests again to check that nothing else broke.
- Check whether the same mistake exists elsewhere in the program. A developer who made it once has probably made it again.
- Fix the real cause, not only the visible symptom. Patching the place where the bug appears leaves its cause free to show up somewhere else.
From the slides: the Pareto technique. At the end of the lecture the professor mentioned the Pareto technique from the debugging slides. It is the 80–20 idea of defect clustering (1.5): about 80% of the bugs are found in about 20% of the code, so effort spent where bugs have already clustered pays off most.
Worked Examples
Exercise 1
What is the difference between testing and debugging? Who does each one? Which part of debugging is harder?
Show solution ↓Hide solution ↑
Solution
Testing shows that a bug exists; testers do it. Debugging locates the cause of the bug and fixes it; developers, who own the code, do it.
Locating the cause is the harder part: about 90–95% of the debugging effort, against 5–10% for the fix. This happens because locating the cause needs a deep understanding of the code; once the cause is understood, the fix is usually easy.
Exercise 2
A student writes: “Testing finds bugs and debugging fixes them.” What would the professor correct, and why?
Show solution ↓Hide solution ↑
Solution
The professor would correct the word “find”. To find something means to know where it is, but testing does not tell you the location. A bug can appear in one place while its real cause is somewhere else, earlier in the program.
The correct statement is: the goal of testing is to show that the software has bugs; the goal of debugging is to locate the cause and fix it. Locating is part of debugging, and it is the larger part.
Exercise 3
Why can a bug that testers found stay unfixed?
Show solution ↓Hide solution ↑
Solution
This happens because of three reasons:
- Cost and time: a fix may be too expensive, or too late (for example, the release is tomorrow).
- Low risk: the bug is minor and rare, so fixing it is not worth the cost.
- New bugs: the fix itself may create new bugs, so changing the code is a risk of its own.
Exercise 4
What is the difference between debugging by induction and by deduction?
Show solution ↓Hide solution ↑
Solution
Induction goes from specific clues to a general theory. We collect the data, look for a pattern, form a theory of the cause, and prove it. Example: the phone rang in the kitchen and the dog was there, so the theory is that the dog moved it.
Deduction goes from a full list of possible causes to one cause. We list every possible cause, use tests to cross out the wrong ones until one is left, and then prove it. Example: list bag, kitchen, car, bedroom; the phone rings inside the house, so cross out the car; continue.
The decisive difference is the direction: induction builds up a theory from the clues, deduction narrows down a complete list. Induction fails if the clues are incomplete; deduction fails if the list is incomplete.
Exercise 5
Name the debugging strategy in each case:
(a) A developer adds System.out.println after almost every line of a large program and reads thousands of lines of output.
(b) A function returns the wrong total; the developer starts at the return statement and reads upwards until the value first goes wrong.
(c) A developer writes three new unit tests, each feeding a different stage of a pipeline, to see where the wrong value first appears.
(d) A developer lists five possible causes and rules them out one at a time with experiments until one remains.
Show solution ↓Hide solution ↑
Solution
- (a) Prints everywhere with no plan: brute force.
- (b) Start at the wrong output and go backwards: backtracking.
- (c) Extra test cases written to pinpoint where the bug starts: debugging by testing.
- (d) Full list of causes, cross out the wrong ones: deduction.
Exercise 6
A developer fixes a crash on the checkout screen by adding a check that hides the error message. State the three rules for fixing a bug and say which one this developer broke.
Show solution ↓Hide solution ↑
Solution
The three rules are:
- After the fix, run regression tests (the old tests again) to check nothing else broke.
- Check whether the same mistake exists elsewhere in the program.
- Fix the real cause, not only the visible symptom.
Hiding the error message removes the symptom while the cause of the crash remains, so the developer broke rule 3. The cause can resurface elsewhere, and without regression tests (rule 1) nobody would notice.
Exercise 7
The slides give two numbers from the IBM Rochester study: ×9 and ×13. What do they mean, and what lesson do they support?
Show solution ↓Hide solution ↑
Solution
A bug that slips from the coding phase into the testing phase costs about 9 times more to fix, and one that slips from testing to real customers costs about 13 times more. Together they support the economics of testing from 1.3: the later a bug is found, the more it costs, so testing should start early. (The slides also note that debugging can take up to 50% of a developer’s coding time.)