Theory: The STLC, Defects & the Economics of Testing
Development has a life cycle, and so does testing. This lesson walks through the Software Testing Life Cycle (STLC), then asks two questions the professor kept returning to: what exactly counts as a bug? and why does the moment you find it matter so much? The answers explain why testers start work long before any code exists.
The STLC: Testing Has a Life Cycle of Its Own
The Software Testing Life Cycle (STLC) is the sequence of activities carried out to make sure the quality goals are met. Its main stages are:
#
Stage
What happens
1
Requirement analysis
Study what must be tested, using the SRS.
2
Planning
Decide the approach, and which combination of test techniques to use.
3
Test design
Actually produce the test cases (input + expected result).
4
Environment set-up
Build a safe place to test, kept apart from the live production system.
5
Execution
Run the tests and compare actual with expected results.
6
Closure
Finish this round: report, archive, learn.
Diagram loads as you scroll.
Why planning tells you something
The professor used the planning stage to make a point. Nobody writes a plan to make toast: planning costs time, and for a trivial job the plan would cost more than the job itself. So if testing deserves a planning stage of its own, testing must be complicated. It needs good tools, good knowledge and good preparation; you should never charge at the software headfirst. During planning you already think about which techniques you will use and, since a real application never yields to a single technique, which combination of them.
A few later stages deserve a remark:
Environment set-up keeps your experiments from damaging the live product. If you work with several versions of the software, you also need to track which tests were run against which version, which in real projects is complicated on its own.
The loop matters. The STLC and the SDLC move hand in hand. When development starts a new iteration, the requirements change, so the analysis changes, so testing has extra work to do. A change on one side produces a change on the other.
Defects, and Why “Feature” Is a Scary Word
Software people have a whole vocabulary for things that are wrong: bug, defect, fault, problem, error, incident, anomaly, variance, failure, inconsistency, and even feature. In this course the professor treats nearly all of them as one thing and simply says bug, except where the difference matters (you will meet one such place in Lecture 8, where JUnit forces you to tell a failure from an error).
How can a feature be a bug?
Imagine you are a developer. The specification says nothing about a dark mode, but you think the customer would love one, so you quietly add it. It is kindly meant. But the tester, checking your software against the specification, sees behaviour the specification does not mention, and reports it as a bug.
From a tester’s point of view, an unplanned extra is a defect. The professional way to add something helpful is to update the specification first, so that everyone agrees on what the software is supposed to be.
The gap between ideal and real
The professor drew a picture that makes the whole idea visible. Draw a perfect circle: the ideal software everyone on the project imagines. Now draw over it the software that really gets built. The two never line up exactly.
Whatever does not overlap is a bug: the missing part and the extra part.
Where the ideal circle sticks out, something is missing.
Where the real software sticks out, something extra was built that nobody asked for.
The overlap is the part you got right.
The gap is normal. It is not the sign of a bad team; it is always there, and it is exactly the reason testing exists.
Where Bugs Are Born
Most beginners assume bugs are mostly typing mistakes in code. Research says otherwise:
Where the defect is created
Share of all defects
Requirements & specification
55%
Design
25%
Code
15%
Other (faulty compilers, hardware, support systems)
5%
Add the first two rows and you get the sentence the slides highlight:
More than 80% of bugs exist before a single line of code is written.
Think of a recipe. If it says “add a cup of salt” by mistake, no amount of careful cooking rescues the dish, because the mistake was made before anyone switched on the stove. Most software problems are like that: a wrong or missing idea, made early, then faithfully carried out later.
The Economics of Testing: ×10 per Stage
Now suppose you catch the salt mistake:
just by reading the recipe: cross out one word, cost about $1;
after buying the ingredients: the shopping is wasted, about $10;
after cooking: a whole meal is wasted, about $100;
after serving a hundred guests: $1,000 or more, plus a bad review that follows you around.
Every time a bug travels one stage further down the line, the cost of dealing with it grows by roughly ten times.
The same bug costs about ten times more at every stage (bars on a squeezed scale).
This is the economics of testing, and it is the reason behind the principle “test early” (Principle 3 in 1.5). But how do you test a recipe, or a specification, when there is nothing to run? You have people read it, question it and challenge it. That is a review, and Lecture 2 (2.6) teaches the different kinds.
From the slides: rework. Rework accounts for about 44% of total development cost. Rework simply means redoing work because an earlier mistake was found late. It is the money side of the same lesson: catching mistakes early, through reviews, saves more than it costs.
Common mistake: thinking testing starts when the code is ready. Since about 80% of defects are created in requirements and design, a tester who waits for code finds most bugs at the expensive end of the ×10 ladder.
Worked Examples
Exercise 1
List the six stages of the STLC in order. Then explain what the existence of a planning stage tells you about testing.
Show solution ↓Hide solution ↑
Solution
The stages are: requirement analysis → planning → test design → environment set-up → execution → closure.
Planning has a cost of its own. We only plan jobs that are complicated enough for the plan to pay for itself (nobody plans to make toast). So a dedicated planning stage tells us that testing is complicated: it needs tools, knowledge and preparation, and a deliberate choice of which combination of test techniques to use.
Exercise 2
A developer adds a “dark mode” that the specification never mentions, because users would probably like it. The tester reports it as a bug. Is the tester right? What should the developer have done?
Show solution ↓Hide solution ↑
Solution
Yes, the tester is right. The tester checks the software against the specification. Behaviour the specification does not mention is an extra, the part of the built software that sticks out beyond the ideal, and from a tester’s point of view an unplanned extra is a defect, however kindly meant.
The developer should have updated the specification first (proposed the feature, had it agreed), so that everyone agrees on what the software is supposed to do. Then it is tested like any other requirement.
Exercise 3
About what share of all defects are created before any code is written? Which two phases account for them?
Show solution ↓Hide solution ↑
Solution
Requirements and specification create about 55% of defects, and design about 25%:
55%+25%=80%
So more than 80% of defects exist before a single line of code is written, which is why testers review requirements and designs instead of waiting for code.
Exercise 4
A bug found in the design phase costs $10 to fix. Roughly what would it have cost in the specification phase, and in production?
Show solution ↓Hide solution ↑
Solution
Each stage multiplies the cost by about 10, so we divide by 10 going back and multiply going forward:
Production (released): design → coding → released is two more stages, 10×10×10=about $1,000 or more.
In production the real cost also includes reputation and legal damage, so $1,000 is a lower bound.
Exercise 5
The slides say rework is about 44% of total development cost. What is rework, and how does it connect to the “test early” principle?
Show solution ↓Hide solution ↑
Solution
Rework means redoing work because an earlier mistake was found late. Because a defect costs about ten times more at every stage it survives, a mistake found late forces expensive rework of everything built on top of it. Catching mistakes early, for example through reviews of the specification and design, removes them while they are still cheap, so the reviews save more than they cost. That is exactly the “test early” principle.