About

1.1 Testing in the Age of AI & the SDLC Map

Theory: Testing in the Age of AI & the SDLC Map

The professor did not open the course with a definition. The first lecture opened with a worry: what generative AI is doing to the software job market, and what that means for the skills you should build. That worry explains why every later topic in this course is taught the way it is. Once it is clear, the lesson draws the map the whole course keeps coming back to, the Software Development Life Cycle (SDLC), and shows why testing belongs at every point on it.

Why This Course Looks the Way It Does

Imagine a bakery. Ten years ago, to bake a thousand loaves in one morning, it needed ten bakers at ten tables. Today a machine kneads and bakes, quickly and cheaply. The bakery has not closed, and it still hires people, but it hires a very different kind of person:

Generative AI is that machine for software. You describe a program in plain language and within minutes it produces the code. It will even write the test cases and run them. Work that used to take a small team months can now be done by one person in about a week.

Cheaper software, fewer jobs

The professor spelled out what follows. When software becomes quicker to make, it becomes cheaper to sell. A small project that once earned a company two or three hundred thousand may now earn less than a hundred thousand, and it needs only one developer. Less income means fewer people the company can afford, and the people it keeps must clearly offer something the machine does not.

In about a year you graduate, and so does every other computer science student. Nearly all of them will know how to type a request into an AI and get something back, so that skill alone will not make anyone stand out. On top of that, experienced developers who were laid off will be competing for the same jobs. The professor was honest that the picture is not entirely bleak, but wanted the class to adjust while there is still time.

What companies actually ask for

The professor said that companies gave the same answer again and again. You must be able to prompt correctly and efficiently, and above all you must be able to verify the result: look at what the AI produced and judge its quality.

And here is the twist that shapes the whole course: that judgment does not come from writing huge amounts of code. It comes from theory, the principles and definitions that feel dull to study but let you say exactly why something is wrong, and that also let you write better instructions for the AI. The companies’ advice was to spend a little less energy on typing code, since AI will out-type you anyway, and a little more on foundations.

Where Humans Still Fit in Testing

The professor also sketched where people fit in the world of testing:

Kind of testingWho does it bestWhy
Anything where code exists and a test can be generated and run automaticallyAI / automationAutomation is quick, efficient and cheap.
User experience (UX) testing: does a real human find the software pleasant to use?A human in the loopIt is about real people’s reactions, which a machine cannot feel.
Exploratory testing: no script, no framework; the tester wanders through the software looking for trouble, guided by curiosityA human in the loopIt runs on curiosity and judgement, not on a fixed procedure.

Even in the human areas you can work with the AI, for example by asking it to play the role of a certain type of user (a persona) and to propose test ideas from that persona’s point of view. Keep this in mind whenever a later topic feels abstract: the purpose is not to compete with the machine, it is to give you the vocabulary to control it.

From the slides: course facts. The course is graded 25% on attendance, participation, in-class exercises and homework, and 75% on the final exam.

The SDLC Map: Building a House

Every software course draws the SDLC: requirements, analysis, design, implementation, testing. The professor drew it again on purpose, because almost every idea in testing can be pinned to a place on it. Walk it with a house:

SDLC stageThe houseThe software
RequirementsThe family says, “We want a home for five people, with a big kitchen.” Everyday language; leaves out rooms, sizes, materials.The user’s needs, in everyday language. Very abstract.
AnalysisAn architect studies what the family said, removes contradictions and unclear parts, and writes a precise description of what the house must be.The Software Requirements Specification (SRS).
DesignThe detailed blueprint that finally picks materials and technologies.The design: components, technologies, structure.
ImplementationThe actual building.The code.

Exam radar: if the exam asks what the typical output of the analysis stage is, the answer is the SRS (Software Requirements Specification).

Why Analysis Stays Free of Technology

Students often stumble on one detail. The professor asked: why should the result of analysis be independent of any technology?

The answer is separation of concerns. The problem you are solving hardly changes over the years, but technology changes constantly: new libraries, new frameworks, better algorithms. If you glue your solution to a specific technology too early, every technology change forces you to untangle the whole thing. So you first decide, in an abstract way, which components the solution needs. Only during design do you decide which components will be built in-house, which will be bought, and which technology each one will use.

From Abstract to Concrete: Four Views of One System

More abstractMore concreteUser needseveryday languageSRSprecise, no technologyDesigntechnology chosenCodethe running systemTTTTreview itreview itreview itrun itAll four describe the SAME system. Testing (T) belongs at every stage.
From abstract to concrete: four descriptions of one system, each of which can be tested.

Moving from left to right, the description becomes more concrete. Yet all four, the user’s needs, the SRS, the design and the code, describe the same system, only at different levels of detail. If everybody did their job well, the four should agree perfectly, like four maps of the same city drawn at four zoom levels.

This is exactly why testing is not only about code. A tester can check the requirements, the specification and the design too: you review the first three and run the last. Anything that describes the system can be tested, and all such things are called software artifacts.

One more point about the picture. We draw the SDLC as a straight line, but real projects go around it again and again, in iterations. So testing is not a final station at the end of the line; it is mixed into every stage. If you wait for the last stage, it is too late and too expensive (the numbers that prove it are in 1.3).

Common mistake: treating “testing” as the last box of the SDLC, done only on code. Requirements, the SRS and the design are software artifacts too, and testing them early is what keeps the cost of bugs down.


Worked Examples

Exercise 1

The professor compared generative AI to a machine in a bakery. Explain the comparison, and state the two skills companies said they want from new graduates.

Show solution ↓
Solution

A bakery that once needed ten bakers now uses a machine that kneads and bakes quickly and cheaply. It still hires people, but different ones: people who can spot a loaf that looks perfect outside and is raw inside, and people who can describe the order precisely so the machine bakes the right thing. Generative AI is that machine for software: it writes code, and even tests, in minutes.

So the two skills companies asked for are:

  1. Prompting correctly and efficiently (describing the order precisely), and
  2. above all, verifying the result: judging the quality of what the AI produced (hallucinations, missing parts, whether it meets the requirements).
Exercise 2

Why did the professor argue that, in the age of AI, studying testing theory matters more than typing more code?

Show solution ↓
Solution

Verifying AI output means saying exactly why something is wrong: that it hallucinated, left something out, or fails a requirement. That judgement does not come from writing large amounts of code; it comes from the principles and definitions of testing, which give you precise words and criteria to judge with. The same theory also helps you write better instructions for the AI.

Meanwhile, typing code is exactly what the AI already does faster than any person. So the companies’ advice was to spend a little less energy on typing code and a little more on foundations, because that is the skill the machine does not replace.

Exercise 3

Name two kinds of testing where a human in the loop gives much better results than automation, and explain why. How can AI still help there?

Show solution ↓
Solution
  1. User experience (UX) testing asks whether a real human finds the software pleasant to use. Only a person can honestly report that experience.
  2. Exploratory testing has no script and no framework: the tester wanders through the software looking for trouble, guided by curiosity and judgement rather than a fixed procedure.

Automation shines wherever a test can be generated and run automatically against code; it struggles wherever a real person’s reactions are the point. Even so, AI can help: you can ask it to play a persona, a certain type of user, and propose test ideas from that persona’s point of view.

Exercise 4

What is the typical output of the analysis stage of the SDLC? Use the house analogy to explain how it differs from the requirements.

Show solution ↓
Solution

The output of analysis is the Software Requirements Specification (SRS).

In the house analogy, the requirements are the family’s words: “a home for five people, with a big kitchen.” They are in everyday language and leave out nearly everything. The architect then studies them, removes contradictions and unclear parts, and writes a precise description of what the house must be. That precise document is the SRS. It is more concrete than the requirements but still says nothing about materials or technology; those are chosen later, in design.

Exercise 5

(Discussion) Discuss why the result of the analysis stage should be independent of any particular technology.

Show solution ↓
Solution

Principle. The relevant principle is separation of concerns: the problem being solved and the technology used to solve it are kept apart. The problem hardly changes over the years, while technology (libraries, frameworks, algorithms) changes constantly.

What it means here. If the analysis glued the solution to a specific technology, every technology change would force the team to untangle the whole solution. Instead, analysis decides in an abstract way which components the solution needs, and only the design stage decides which are built in-house, which are bought, and which technology each uses. This happens because the stable part (the problem) should not depend on the unstable part (the technology).

Exercise 6

A classmate says: “Testing is the last stage of the SDLC. You cannot test anything until the code is written.” Is the classmate right? Explain using the abstract-to-concrete picture.

Show solution ↓
Solution

No. The user needs, the SRS, the design and the code are four descriptions of the same system at different levels of detail. Each of them is a software artifact, and any artifact can be tested: the first three by reviewing them, the code by running it.

Also, real projects go around the SDLC in iterations, so testing is mixed into every stage rather than saved for the end. Waiting for code means finding problems at the most expensive point (see 1.3).