Theory: Static vs. Dynamic Testing & Myers’ Ten Principles
Lecture 3 is the lecture where testing turns into a craft: by its end you will design tests from code and build the MC/DC tables the professor said appear very often in the exam. Before any of that, the lecture settles three groundwork questions. Does testing always mean running the program? Since we cannot find every bug, which ones do we look for first? And what rules of thumb do experienced testers live by?
Does Testing Always Mean Running the Program?
No. In Lecture 2 (2.6) a requirements document was tested simply by reading it. A document cannot run, yet it was tested. So there are two kinds of testing.
Think about buying a used car:
- You can take it for a test drive: start it, drive it, and see how it behaves.
- You can look under the hood before you ever start the engine: check the oil, the belts, the rust.
Both are ways of checking the car, and they find different problems.
- Dynamic testing is the test drive. You run the program with some input, then compare the actual result with the expected result. Unit, integration, system and acceptance tests are all dynamic.
- Static testing is looking under the hood. You check without running anything: you read code, check a design, or review a requirements document.
| Static testing | Dynamic testing |
|---|
| Runs the program? | No | Yes |
| What you do | Read, inspect, review | Give input, compare actual with expected |
| Typical examples | Requirements reviews, design reviews, code reviews | Unit, integration, system, acceptance tests |
| Can start before code exists? | Yes | No |
Why the difference matters
Recall from 1.3 that more than 80% of bugs are created in the requirements and the design, before any code exists. There is nothing to run at that point, so static testing is the only way to catch those bugs early, when they are cheapest to fix.
From the slides: static testing tools. Static testing is not only done by people. Linters and static analysers (for example SonarQube) read your code without running it and warn you about suspicious lines. The red and yellow underlines in your code editor are a small version of the same idea.
We Cannot Find Every Bug. Which Bugs Come First?
The professor asked the class: finding every error is impossible, so what should a tester assume?
Some students answered “we should try to break the program”. That is true, but it answers a different question: breaking the program is the purpose of testing, not the assumption a tester makes. The professor warned that a true answer to a different question gets no marks. Read what the question actually asks.
The answer builds up step by step:
- We cannot find all errors, so we will only find some of them.
- So we must choose which ones to look for.
- Errors are not equally important, because features are not equally important.
Think of a banking app. A button in the wrong colour is a small problem. Money sent to the wrong account is a disaster. A tester with limited time should obviously spend it on the money transfer first. That choice is made with risk analysis (2.2): rank the features by how likely and how harmful a failure would be, and hunt for the critical errors first.
Exam radar: Assumption: not all errors are equally important, because not all features are equally important. So the tester uses risk analysis to find the more critical errors first.
The professor’s goal for testers: find as many bugs as possible, in as little time as possible, as early as possible.
Myers’ Ten Principles
These principles come from the classic book The Art of Software Testing by Glenford Myers. They are a different list from the seven ISTQB principles in 1.5, although some of the ideas overlap (for example, “where you found many bugs, more are hiding” is defect clustering again).
| # | Principle | What it means |
|---|
| 1 | A test case must include the input and the expected result. | Without an expected result you cannot say whether the answer is right. |
| 2 | A programmer should avoid testing his or her own program. | The owner wants the program to work, so the tests become gentle. Exceptions: unit testing and debugging. |
| 3 | A programming organisation should not test its own programs. | The same bias, at the level of a whole company. |
| 4 | Inspect the result of every test carefully. | Also true for tests that an AI writes and runs for you. |
| 5 | Write test cases for invalid and unexpected input, not only valid input. | These are negative tests. |
| 6 | Always use the definitions of a bug when you check the output. | The five definitions from 1.6. |
| 7 | Avoid throwaway test cases. | Keep them so you can run them again later (regression testing). |
| 8 | Do not plan testing as if no errors will be found. | There are always errors. Plan to find them. |
| 9 | Testing is creative and intellectually challenging. | Good tests need imagination, not only rules. |
| 10 | The more errors you find in one part, the more are likely to be there. | Murphy’s Law, the same idea as defect clustering. |
The ones the professor talked about
The professor did not read the list out line by line. These five got real attention in class:
- Input and expected result (1). A test case is not just “try 5”. It is “try 5, expect 25”. The expected result is what we compare against; without it there is no verdict.
- Do not test your own program (2). It is like marking your own exam: you know what you meant to write, so you read what you meant instead of what is there. The two exceptions are unit testing and debugging, which the owner does best because they need intimate knowledge of the code.
- Check every result carefully (4). The professor stressed that this matters even more now that an AI can write and run tests for you. A test that “passed” is only as good as the person who checked what it actually compared.
- Test wrong and unexpected input too (5). Users type letters into number fields and press buttons in the wrong order. Tests with invalid input are called negative tests (you will build them systematically in 5.3).
- Where you found many bugs, more are hiding (10). Murphy’s Law: if a module has already produced many bugs, it is probably badly built, so dig deeper there.
Common mistake: mixing up Myers’ ten principles with the seven ISTQB principles of 1.5. If the question names Myers, answer with Myers’ list; if it asks for “the seven principles of testing”, use the ISTQB list.
Worked Examples
Exercise 1
What is the difference between static and dynamic testing? Give one example of each.
Show solution ↓Hide solution ↑
Solution
Dynamic testing runs the program with some input and compares the actual result with the expected result, like a test drive. Example: a JUnit unit test that calls a method and checks its return value.
Static testing checks without running anything, like looking under the hood. Example: a review of the requirements document, or a code review.
An important consequence: static testing can start before any code exists, so it can catch requirements and design bugs at the cheap end of the ×10 cost ladder.
Exercise 2
Finding every error is impossible. What should a tester assume? A student answers: “The tester should assume the program is broken and try to break it.” Would this answer earn the marks?
Show solution ↓Hide solution ↑
Solution
The expected answer: not all errors are equally important, because not all features are equally important. So the tester uses risk analysis to find the more critical errors first.
The student’s answer would not earn the marks. “Try to break the program” is true, but it describes the purpose of testing, not the assumption the question asks for. The professor’s rule is that a true answer to a different question gets no marks.
The reasoning behind the correct assumption: since we can only find some errors, we must choose which ones; and since a wrong button colour matters far less than money sent to the wrong account, we choose by risk.
Exercise 3
Classify each activity as static or dynamic testing:
- SonarQube scans the repository and flags an unused variable.
- A tester types a negative age into a sign-up form and checks the error message.
- Three colleagues read the design document and question its assumptions.
- A JUnit test suite runs after every commit.
Show solution ↓Hide solution ↑
Solution
Ask one question each time: is the program running?
- Static: a static analyser reads the code without running it.
- Dynamic: the program runs and its output is compared with the expected error message (this is also a negative test).
- Static: a review of a document; nothing runs.
- Dynamic: the unit tests execute the code.
Exercise 4
Myers says a programmer should avoid testing his or her own program. Explain why, and name the two activities that are exceptions.
Show solution ↓Hide solution ↑
Solution
The owner of a program wants it to work. That wish quietly shapes the tests: they become gentle, avoiding the inputs most likely to break the code. It is like marking your own exam; you read what you meant to write, not what you actually wrote. The same bias exists at company level, which is why Myers adds that an organisation should not test its own programs.
The two exceptions are unit testing and debugging. Both need detailed knowledge of the code’s internals, which the owner has better than anyone.
Exercise 5
A tester writes the test case “enter the value 17 into the age field”. According to Myers’ first principle, what is missing, and why does it matter?
Show solution ↓Hide solution ↑
Solution
The expected result is missing. A test case must contain both the input and the expected result, for example “enter 17 → expect the message You must be 18 or older”.
Without the expected result, the tester has nothing to compare the actual output against, so the test can never pass or fail: it only shows what happened, not whether it was right.
Exercise 6
During testing, module Payments has produced 14 bugs and module Profile has produced 1. You have time for one more day of testing. Which module should get it? Name the principle you are using.
Show solution ↓Hide solution ↑
Solution
Payments. Myers’ tenth principle (Murphy’s Law) says the more errors you find in one part, the more are likely to remain there; many bugs suggest the module is badly built, so more are hiding. This is the same idea as defect clustering in the ISTQB principles.
Risk analysis points the same way: payments are also the more critical feature, so a bug there costs far more than one in a profile page.