7.3 Category 3: The Slope of the Bug Curve

Theory: Category 3: The Slope of the Bug Curve

Category 3 needs no history. Instead of asking “what happened on old projects?”, it asks “what is happening on this project right now?” You record how many bugs you have found as testing goes on, and you stop when finding them suddenly becomes much harder. This lesson shows how to draw the bug curve, how to measure its slope, and exactly where the professor’s rule tells you to stop.

Drawing the Bug Curve

Think of picking apples from a tree. In the first hour you fill basket after basket from the low branches. Later you are stretching for the odd apple near the top, and each one takes several minutes. Nobody waits until the tree is completely bare; you stop when the effort per apple suddenly jumps.

The bug curve shows the same thing for testing:

• x-axis: time, in regular time units (for example, one week each);
• y-axis: the total (cumulative) number of bugs found so far.

At first the curve is steep, because many bugs are found every week. Later it flattens, because few new bugs turn up.

The Slope

The slope of a piece of the curve tells you how many bugs were found per time unit in that interval:

$$S = \frac{n_2 - n_1}{t_2 - t_1}$$

where $n$ is the cumulative number of bugs and $t$ is the time. With one-week intervals, $t_2 - t_1 = 1$, so the slope is simply the number of bugs found that week.

• A high slope is good news: many bugs per week, so testing is paying off. Keep testing.
• A slope near 0 means almost no new bugs: testing is no longer paying off.

When Do We Stop?

The obvious answer, “stop when the slope reaches 0”, is a trap: the curve approaches flat only slowly, and waiting for it could take a very long time. The professor’s rule is:

Stop as soon as you see the greatest drop in the slope. Calculate the slope of every interval, then the difference between each slope and the one before it. The biggest negative difference is the greatest drop; stop at the end of that interval.

The weekly example

Week	0	1	2	3	4	5	6
Total bugs found	0	40	85	120	135	142	146
Slope $S_i$ (bugs that week)	–	40	45	35	15	7	4
Change $S_i - S_{i-1}$	–	–	+5	−10	−20	−8	−3

Each slope is this week’s total minus last week’s total, for example $S_4 = 135 - 120 = 15$. Each change compares neighbouring slopes, for example $S_4 - S_3 = 15 - 35 = -20$. The most negative change is −20, between week 3 and week 4, so we stop at the end of week 4, the moment the drop becomes visible.

How to Compute It, Every Time

1. Check the data. Are the numbers cumulative totals, or bugs found per interval? If they are per interval, they are the slopes already.
2. Slopes: for every interval, subtract the previous total from the current one (and divide by the length of the interval if it is not 1).
3. Differences: for every pair of neighbouring slopes, compute $S_i - S_{i-1}$.
4. Greatest drop: pick the most negative difference.
5. Stop at the end of the interval where that drop appears.

Exam radar: if this comes in the exam, the professor will give you the numbers. Show the slope for each interval, the differences between slopes, and name the greatest drop clearly. Remember that this category needs no past data: you monitor the current project.

The Trap: Greatest Drop Is Not the Lowest Rate

Take a second example, measured in hours. The cumulative defects found at hours 0 to 5 are 0, 10, 30, 60, 70, 75.

Hour	0	1	2	3	4	5
Cumulative defects	0	10	30	60	70	75
Slope	–	10	20	30	10	5
Change	–	–	+10	+10	−20	−5

The greatest drop is −20, from 30 defects per hour to 10, so the stopping candidate is hour 4. Hour 5 has the lowest rate (5), but the question asks for the greatest drop, and those are two different calculations.

Common mistake: choosing the interval with the smallest slope, or waiting for the slope to reach 0. The rule is the largest decrease between neighbouring slopes. Also note the early slopes may rise (positive changes) while testers are still getting into the software; those are not drops.

What the Slope Cannot Tell You

A sharp drop in the discovery rate signals diminishing returns: each extra hour of testing now buys far fewer bugs. It is not proof that no defects remain. A falling rate can also mean that the tests themselves have become weak, for example because the team keeps running the same kind of test. That is why a Category 3 decision is best combined with evidence that the planned techniques were applied thoroughly (Category 1 coverage where it can be measured, see 7.1).

---

Worked Examples

Exercise 1

What do the axes of the bug curve show, and what does a high slope or a slope near 0 tell the tester?

Show solution ↓Hide solution ↑

Solution

The x-axis shows time in regular units (for example weeks); the y-axis shows the cumulative number of bugs found so far.

The slope is the number of bugs found per time unit. A high slope means many bugs are being found each period, so testing is paying off and should continue. A slope near 0 means almost no new bugs are being found, so testing is no longer paying off.

Exercise 2

Why does the professor’s rule stop at the greatest drop in the slope instead of waiting until the slope reaches 0?

Show solution ↓Hide solution ↑

Solution

Because waiting for a slope of 0 could take far too long. The curve flattens only gradually, so the last stretch before it becomes completely flat can last a very long time while producing almost no bugs, which is expensive and gives little in return. The greatest drop marks the point where the return on testing changes most sharply: before it, testing was productive; after it, each time unit buys much less. That makes it a sensible, objective place to stop.

Exercise 3

The cumulative number of bugs found at the end of each week of system testing is:

Week	0	1	2	3	4	5	6	7
Total bugs	0	30	70	100	115	122	126	128

Calculate the slopes and the changes in slope. When should system testing stop?

Show solution ↓Hide solution ↑

Solution

Slopes (this week’s total minus last week’s):

$S_1 = 30$, $S_2 = 70 - 30 = 40$, $S_3 = 100 - 70 = 30$, $S_4 = 115 - 100 = 15$, $S_5 = 7$, $S_6 = 4$, $S_7 = 2$.

Changes $S_i - S_{i-1}$:

Interval	2 vs 1	3 vs 2	4 vs 3	5 vs 4	6 vs 5	7 vs 6
Change	+10	−10	−15	−8	−3	−2

The greatest drop is $S_4 - S_3 = 15 - 30 = -15$. Stop system testing at the end of week 4.

Exercise 4

Cumulative bugs found at the end of each day: day 0: 0, day 1: 25, day 2: 45, day 3: 60, day 4: 70, day 5: 74, day 6: 76. Find the stopping point.

Show solution ↓Hide solution ↑

Solution

Slopes: $S_1 = 25$, $S_2 = 20$, $S_3 = 15$, $S_4 = 10$, $S_5 = 4$, $S_6 = 2$.

Changes: $S_2 - S_1 = -5$, $S_3 - S_2 = -5$, $S_4 - S_3 = -5$, $S_5 - S_4 = 4 - 10 = -6$, $S_6 - S_5 = -2$.

The most negative change is −6, between day 4 and day 5. Stop at the end of day 5.

Notice that the slope was falling steadily by 5 every day; the rule picks the interval where it falls most, not the first interval where it falls.

Exercise 5

A test lead reports the number of bugs found in each week (not the running total): 12, 20, 18, 6, 4, 3 for weeks 1 to 6. When should testing stop?

Show solution ↓Hide solution ↑

Solution

These are interval counts, so they already are the slopes: $S_1 = 12, S_2 = 20, S_3 = 18, S_4 = 6, S_5 = 4, S_6 = 3$. (Turning them into cumulative totals, 12, 32, 50, 56, 60, 63, and back gives the same slopes.)

Changes: $+8, -2, -12, -2, -1$.

The greatest drop is $S_4 - S_3 = 6 - 18 = -12$. Stop at the end of week 4.

Exercise 6

The cumulative defects at hours 0 to 5 are 0, 10, 30, 60, 70, 75. A student says: “Stop at hour 5, because that is where the rate is lowest.” Is the student right?

Show solution ↓Hide solution ↑

Solution

No. The slopes are 10, 20, 30, 10, 5 and the changes are +10, +10, −20, −5. The greatest drop is −20 (from 30 to 10 defects per hour), so the stopping candidate is hour 4.

Hour 5 does have the lowest rate, but the Category 3 rule asks for the greatest drop between neighbouring slopes, not the smallest slope. Those are different calculations.

Exercise 7

Your team’s bug-discovery rate has just dropped sharply. A manager says: “Great, the software must be nearly bug-free.” Evaluate this claim.

Show solution ↓Hide solution ↑

Solution

The claim goes too far. A sharp drop in the slope shows diminishing returns: the current tests are now finding far fewer bugs per time unit, so continuing in the same way is not paying off. It does not prove that no defects remain; testing can show the presence of bugs, never their absence.

The drop may even have another cause: the tests may have become weak or repetitive, so they no longer reach the parts of the software where bugs remain. Before relying on the trend, the team should check that its planned techniques were applied thoroughly, for example with coverage measurements (Category 1). The correct conclusion is “this is a reasonable point to stop this kind of testing”, not “the software is bug-free”.