Practice Exam: Test Completion Criteria

Theory: How to Approach These

This practice exam follows the professor’s style for test completion criteria (suggested time: 25 minutes). It combines a Category 2 calculation with a Category 3 slope question. Review these before attempting it:

State your assumption first. If the 40% CLD / 60% design split is not given, write it as an assumption before any calculation (see 7.2).
Category 2 procedure (see 7.2):
1. Total errors = statements × error rate.
2. Split into CLD and design errors.
3. Goals = goal percentage × each kind’s total.
4. Each level = CLD rate × CLD total + design rate × design total (the “65% of what?” trap).
5. Check: sum of level targets = CLD goal + design goal.
Category 3 procedure (see 7.3): slope per interval, difference between neighbouring slopes, stop at the most negative difference (not the lowest slope).
Name the category when asked: Category 2 = past statistics, Category 3 = bug-finding trend (see 7.1).

The exam question. You are responsible for testing a program of 25,000 statements. After code inspections, about 4 errors per 100 statements are expected to remain. The objective is to detect 98% of the coding and logic-design (CLD) errors and 95% of the design errors. Past projects show the following detection rates:

Test level CLD errors found Design errors found
Module test 65% 0%
Function test 30% 60%
System test 3% 35%

During system testing, the total number of bugs found at the end of each week was:

Week 0 1 2 3 4 5 6
Total bugs found 0 40 85 120 135 142 146

State explicitly your assumption(s).

Practice Exam Solutions

Exercise Q1a

Calculate the expected total number of errors.

Show solution ↓

Solution

Assumption: CLD errors are 40% and design errors 60% of all errors (industry study). The error rate of 4 per 100 statements applies uniformly to the whole program.

There are $25{,}000 / 100 = 250$ groups of 100 statements, each with about 4 errors:

\frac{25{,}000 \times 4}{100} = 1{,}000

Expected total: 1,000 errors.

Exercise Q1b

Calculate the expected number of CLD errors and design errors.

Show solution ↓

Solution

Using the 40/60 assumption:

\text{CLD} = 0.40 \times 1{,}000 = 400 \qquad \text{Design} = 0.60 \times 1{,}000 = 600

CLD errors: 400; design errors: 600. Check: $400 + 600 = 1{,}000$ ✓

Exercise Q1c

Calculate the goal (number of errors to detect) for CLD errors and for design errors.

Show solution ↓

Solution

The goals are set below 100% because finding every error is unattainable:

\text{CLD goal} = 0.98 \times 400 = 392 \qquad \text{Design goal} = 0.95 \times 600 = 570

CLD goal: 392 errors; design goal: 570 errors.

Exercise Q1d

Calculate the test completion criterion for the module test, the function test and the system test.

Show solution ↓

Solution

Each detection rate is a percentage of the total of its own kind of error (400 CLD, 600 design), not of 1,000 and not of the goals.

Test level	Calculation	Criterion
Module test	$0.65 \times 400 + 0.00 \times 600 = 260 + 0$	260 errors
Function test	$0.30 \times 400 + 0.60 \times 600 = 120 + 360$	480 errors
System test	$0.03 \times 400 + 0.35 \times 600 = 12 + 210$	222 errors

The module test finds no design errors because it examines one unit at a time and is too low-level to see design mistakes.

Check: the CLD rates add to $65 + 30 + 3 = 98\%$ and the design rates to $0 + 60 + 35 = 95\%$ , matching the goals; and $260 + 480 + 222 = 962 = 392 + 570$ ✓

So: stop the module test after 260 errors have been found (and repaired), the function test after 480, and the system test after 222. These targets count errors found, not test cases run.

Exercise Q1e

Using the weekly data, calculate the slope for each week and the change in slope. When should system testing stop? Which category of test completion criteria is this?

Show solution ↓

Solution

The data are cumulative totals, so each slope is this week’s total minus last week’s (one-week intervals):

Week	1	2	3	4	5	6
Slope $S_i$	40	45	35	15	7	4
Change $S_i - S_{i-1}$	–	+5	−10	−20	−8	−3

The greatest drop is $S_4 - S_3 = 15 - 35 = -20$ . Stop system testing at the end of week 4.

This is Category 3: the bug-finding trend (slope of the bug curve) of the current project. It needs no past data; the stop at week 4 marks diminishing returns, not proof that no bugs remain.