Exam Practice

Software Requirements Specification, the precise, technology-free description of what the system must do, produced by the analysis stage.

Anything that describes the system (requirements, SRS, design, code, test plans, user guides, data). Every artifact can be tested.

Testing with no script or framework; a tester wanders through the software looking for trouble, guided by curiosity. An area where a human in the loop beats AI.

The correct output, decided when the test case is written and before running it, usually derived from the SRS. A test case needs input AND expected result.

Checking a work product against its specification (CMMI). In everyday words, "did we build the thing right?"

Checking that the delivered product fulfils its intended use in its real environment, i.e. the user's real needs (CMMI). In everyday words, "did we build the right thing?"

Software Testing Life Cycle: requirement analysis → planning → test design → environment set-up → execution → closure. It loops with every SDLC iteration.

A bug costs about ten times more at each later stage (about 1 dollar at specification, 10 at design, 100 at coding, 1,000+ after release). Hence "test early".

Redoing work because an earlier mistake was found late; about 44% of total development cost.

Software where a bug can kill or injure people (e.g. radiation therapy machines, aviation); it demands far more testing effort than ordinary software.

Functional = what the software does (save, calculate); easy to check against the spec. Non-functional = how well (performance, security, reliability, usability); needs metrics, tools and special environments.

Load testing gives the system a lot of work; stress testing restricts the resources available to do that work.

Running the same tests repeatedly stops finding new bugs, because the bugs they can reach are already fixed. Keep writing new, different tests.

Bugs gather in the same places (Murphy's Law; roughly 80% of bugs in 20% of the code). Where you find many, look harder.

Professor's reading, not seeing bugs does not mean none exist. ISTQB reading, fixing found bugs is pointless if the system is unusable or misses the users' needs.

1 doesn't do what the spec says it should; 2 does what the spec says it shouldn't; 3 does what the spec doesn't mention; 4 doesn't do what the spec doesn't mention but should; 5 hard to use, slow, or users unhappy.

Verification cannot catch it, because the specification itself is incomplete and the software matches it. Only validation (back to the user's needs) can.

Trying every possible input combination. Even a 32-digit calculator needs at least 2 × 10^64 tests for addition alone (4 × 10^64 if both numbers can be negative), so it is impossible.

The goal of a group of tests (e.g. "test addition"); test cases are written under each objective.

How often a problem can happen combined with how much it costs (to fix, and if not fixed). Test high-impact areas first.

Controllability = you can control the input going in. Observability = you can see the result coming out. Together they make testability.

Three integration problems between units that work alone, namely small numeric errors that grow, messages one unit does not understand, and different rules for talking.

Playing the computer by running the code slowly on paper and writing each variable's value after every step; good for exposing logic errors.

Locating the cause of a bug (90–95% of the effort) and fixing it (5–10%). Done by developers, after testers show the bug exists.

Induction goes from specific clues to a general theory, then proves it. Deduction goes from a full list of possible causes, eliminating until one remains.

Run regression tests after the fix; check whether the same mistake exists elsewhere; fix the real cause, not just the symptom.

After roughly 50–70% of the bugs you will find, the curve flattens and each new bug costs much more effort, where the spiral of unattainable perfection begins.

From casual to strict, walkthrough, ad hoc review, technical review, formal inspection. Formality, preparation and cost grow at each step.

Trained moderator, reader and recorder; a formal process with a checklist, measurement and a step to check that problems were fixed.

Static = checking without running (reviews, linters such as SonarQube). Dynamic = running the program and comparing actual with expected.

A programmer should avoid testing their own program, with the exceptions of unit testing and debugging.

Input-output (data-driven) testing through the interface only. Limited by exhaustive input testing. Done first, to avoid the developer's bias.

Logic-driven (transparent box) testing designed from the code and its paths. Limited by exhaustive path testing.

A decision is the overall true/false outcome of an if; a condition is one atomic true/false check inside it, joined by && or ||.

Every statement runs at least once. The weakest criterion ("baby stuff"), because false branches may never run.

Every decision is true at least once and false at least once. A wrong condition inside a decision can still escape.

Every condition is true at least once and false at least once. Independent of decision coverage; a whole branch can be missed.

In A && B with A false (or A || B with A true), B is never evaluated. A masks B, so a bug in B can stay hidden.

Modified Condition/Decision Coverage. Each condition is shown to independently affect the outcome of the decision. Minimum n + 1 tests.

Two test cases that differ in exactly one condition and give different decision outcomes.

Every column starts with F; the last variable changes every row, the one before every 2 rows, and so on. Rows are numbered 1, 2, 3…

A test set has at least one full pair in every extended column. A minimum test set is a test set with exactly n + 1 rows.

Take a pair, stand on one of its rows, look across to the next extended column, take the pair named there, and repeat, reusing rows to reach n + 1.

Split infinite inputs and outputs into partitions the program treats the same way, then test one sample from each. If it passes, the partition passes.

Range → 1 valid + 2 invalid; specific value → 1 valid + 2 invalid; member of a set → 1 valid + 1 invalid; Boolean → 1 valid + 1 invalid.

For each boundary test the value on it, just below it and just above it (e.g. range 0–75, test −1, 0, 1 and 74, 75, 76).

Testing by an independent outside company, wanted for safety-critical software. Main shortcoming, cost.

A smaller border inside a partition found by thinking deeper, e.g. the position of the equal sides in an isosceles triangle.

Conditions on top, actions below, one rule per column (1, 0, X). A transposed truth table, so 2^K rules for K conditions before don't-cares.

The condition does not affect this rule; one column with X replaces two columns (one with 1, one with 0).

A graph linking causes (inputs) to effects (outputs) through AND, OR and NOT (plus intermediate nodes). Draw it first, then derive the decision table.

NOT(A AND B) = NOT A OR NOT B; NOT(A OR B) = NOT A AND NOT B. "Break the line, change the sign."

E = at most one cause is 1; O = exactly one; I = at least one; R = if A is 1 then B must be 1; M = effect E1 forces effect E2 to 0.

Conditions arranged hierarchically; each root-to-leaf path is one rule and one test case, with "not applicable" for conditions the path never evaluates.

A rule decided in advance that says when testing can stop (also called a stopping criterion).

1 principles & coverage (low-level tests); 2 statistics & past experience (needs prior data); 3 bug-finding trend (stop at the greatest drop in slope).

CLD = coding and logic design errors, about 40% (module tests catch most). Design errors, about 60%, are caught mainly by function and system tests.

S = (n2 − n1)/(t2 − t1), the bugs found per time unit. Stop at the greatest drop between neighbouring slopes.

Independent verification and validation by a separate company, required for safety-critical software. Strength, no bias; weakness, very expensive.

The caller waits for the answer before continuing (like a phone call), which is how a driver usually calls the MUT.

Failure = the test ran but actual ≠ expected (a bug was found). Error = the test could not finish due to an unexpected problem not caused by the test case.

A change a test makes (e.g. to a shared variable) that another test might depend on. Tests must not depend on side effects.

A stub is a fake module below the unit that the unit calls (minimum functionality). A driver is a fake caller above that runs the unit with test inputs.

All the drivers, stubs and set-up around a unit that let you control its inputs and observe its outputs.

Combining tested units step by step and testing them as a group and through their interfaces; catches precision growth, interface errors and timing problems.

Top-down starts at the main module with stubs (early executable, poorer control/observation). Bottom-up starts at the lowest modules with drivers (better control/observation, executable very late).

Write a failing test (red), write the simplest code to pass it (green), clean the code with all tests still passing (refactor), and repeat.

Three ways to green, namely return a hard-coded value, write the simple real code directly, or generalise only when two or more examples require it.

Cleaning code without changing what it does, in three steps (freeze the features, refactor, run a regression test).

Set-up (e.g. in @Before) that creates the objects and initial values every test needs, so each test starts from the same known state.

@BeforeClass once → for each test: @Before → @Test → @After → @AfterClass once. JUnit 5: @BeforeAll, @BeforeEach, @AfterEach, @AfterAll.

Dummy fills a parameter; stub gives canned answers; spy is a stub that records calls; mock expects specific calls; fake is a simple working version (in-memory DB).