The test pyramid says: write mostly unit tests, fewer integration tests, a few end-to-end tests. That instruction means something only if "unit test" does. It has at least five definitions in circulation, and they disagree. Count one suite under two of them and you can get a pyramid and a diamond from the same tests.

So "do we follow the pyramid?" has no answer until a team picks a definition. And the problem the pyramid was meant to solve, a slow and flaky suite, doesn't need the pyramid at all. You can see it directly, and fix it directly.

Five definitions of "unit test"

1. A test of one small piece of the design. IEEE's glossary defines unit testing as "Testing of individual hardware or software units or groups of related units", and admits in a note that the relationship between "module", "component" and "unit" "is not yet standardized"1. Software Engineering at Google says "Units can be as small as an individual function or as broad as a set of several related packages/modules"2.

2. A test with every collaborator replaced by a double. Martin Fowler calls these solitary tests: "mockists insist upon solitary unit tests, while classicists prefer sociable tests"3.

3. A test that touches nothing outside its process. Michael Feathers: "A test is not a unit test if: It talks to the database / It communicates across the network / It touches the file system / It can't run at the same time as any of your other unit tests / You have to do special things to your environment (such as editing config files) to run it"4. Google's testing blog: "A Small test equates neatly to a unit test"5.

4. A test that no other test can affect. Kent Beck's draft of Test-Driven Development: By Example says "tests should be able to ignore each other completely"6. Fowler recalls that "when xunit testing began in the 90's we made no attempt to go solitary unless communicating with the collaborators was awkward"3.

5. A test of one behaviour through a public interface. Roy Osherove's unit is "a single logical functional use case in the system that can be invoked by some public interface (in most cases)"7. Ian Cooper: "the unit of isolation is the test not the thing under test"8.

Even ISTQB, the body behind the most common tester certification, hasn't settled it. Its syllabus says "Component testing (also known as unit testing)"9. Its glossary dropped "unit testing" as a synonym in August 2025, then brought it back in April 2026 as a separate term: "the smallest part of code testable in isolation"10, without saying what isolation means.

Fowler's verdict is that the term is "very ill-defined", and "really it's a situational thing - the team decides what makes sense to be a unit"3. That's reasonable advice. The trouble starts when the team never decides.

The same test, different layers

Take a few ordinary tests and file them under each definition:

Test Small piece All doubles No I/O Independent One behaviour
A pure function, called directly unit unit unit unit unit
A class with its real in-memory collaborators depends integration unit unit unit
A class with every collaborator mocked unit unit unit unit arguably not
An HTTP handler, only the database faked integration integration unit unit unit
A function that writes a temp file unit ? integration unit unit

Only the pure function is a unit test under every definition. Everything else moves between layers depending on who files it, and nothing in the code changes.

One suite, two shapes

Fabian Trautsch and Jens Grabowski took 10 open-source Python projects and classified their tests against the IEEE and ISTQB definitions. Their conclusion: "developers believe that they are developing more unit tests than they actually do"11. Depending on the project and the definition, the share of real unit tests ranged from
0% to 68.30%.

aws-cli shows how much the definition matters. Its developers labelled about 100 tests per revision as unit tests. Under IEEE's definition about 60% of those qualified; under ISTQB's, about 3%11. The tests are the same in both counts; only the definition changes.

The tests that don't qualify don't disappear. The study only says they aren't unit tests, but tests written as unit tests rarely drive the whole system end to end, so they belong in the middle layer. Under IEEE's definition, aws-cli keeps a wide base. Under ISTQB's, it has a thin base and a wide middle. Add any end-to-end tests on top and the same suite is a pyramid under one standard and a diamond under the other.

Neither drawing is wrong. The pyramid simply has no shape until someone says what goes in the bottom layer, and most teams never do.

You don't need the shape to find slow tests

The pyramid's case was economic: unit tests are fast and reliable, so have many of them. But speed is not something you infer from a diagram. Every test runner reports how long each test takes. Sort by duration and the slow tests are at the top, whatever folder they live in and whichever definition they would satisfy.

Look at why they're slow and the answer is almost never "it covers too much code". It's what the test touches: a real database, the network, the disk, a sleep. And tests are flaky for a related reason: they share something, a database, a file, a global, the clock, so the result depends on what ran before or alongside them.

Fix it directly

Two of the five definitions are really engineering rules, and they're the two that matter for speed:

  • Make every test independent (definition 4). A test that sets up its own state and shares nothing can run in any order and in parallel. That alone removes a whole class of flakiness and lets you add cores instead of waiting.
  • Stub the slow parts (definition 3). Replace the database, the network and the clock with in-process fakes where the test isn't about them. Fake the expensive edge, not every collaborator.

Neither rule says how much code a test may cover. An HTTP handler test with the database faked runs in one process, in milliseconds, in parallel with everything else. Under two of the five definitions it's an integration test; for speed it doesn't matter. Cooper makes the same point about TDD: "it's the isolation of the test that is driving
that not the isolation of the code under test"8.

The other argument for small tests is that a failure points at the bug. Trautsch's thesis found that in the 27 projects it studied: "unit tests are better able to pinpoint the source of a defect"12. How much that matters depends on the application, not on the test. If an error surfaces with a clear stack trace, nothing swallows it, there isn't much indirection between the call and the fault, and you can run the test locally under a debugger, a failing integration test is quick to trace. If errors get caught and re-raised as something generic, small tests do most of the pointing. Either way it's a question of debuggability, not of counting layers. The follow-up study found no difference in what the two kinds of test catch: "We could not find any evidence that one test type is more capable of detecting certain defect
types than the other one"13.

So keep the pyramid if your team has defined "unit" and finds the picture useful. If it hasn't, skip the argument about the shape. Measure which tests are slow and flaky, make them independent, and stub what makes them slow.

I keep a knowledge base on software quality where every claim has a source. Terminology maps unit, integration and end-to-end onto Google's scope and size. The axes covers what the pyramid doesn't ask: where the inputs come from and what decides pass
or fail.

  1. IEEE Standard Glossary of Software Engineering Terminology, IEEE Std 610.12-1990, entries "unit testing" and "unit". ↩

  2. Titus Winters, Tom Manshreck, Hyrum Wright (eds.), Software Engineering at Google, O'Reilly, 2020, chapter 12, "Unit Testing". ↩

  3. Martin Fowler, "Unit Test", bliki, 2014 (revised 2017). ↩

  4. Michael Feathers, "A Set of Unit Testing Rules", Artima, 2005. ↩

  5. Simon Stewart, "Test Sizes", Google Testing Blog, 2010. ↩

  6. Kent Beck, Test-Driven Development: By Example, "Isolated Test" (pre-publication draft, 2002; the printed edition may word it differently). ↩

  7. Roy Osherove, "Unit test: a definition", The Art of Unit Testing companion site. ↩

  8. Ian Cooper, TDD, Where Did It All Go Wrong?, talk, DevTernity 2017 (first given 2013); quoted from the recording. ↩

  9. ISTQB, Certified Tester Foundation Level Syllabus v4.0.1, 2024, §2.2.1. ↩

  10. ISTQB Glossary, "unit testing", read November 2026. ↩

  11. Fabian Trautsch, Jens Grabowski, "Are there any Unit Tests? An Empirical Study on Unit Testing in Open Source Python Projects", ICST 2017, doi:10.1109/ICST.2017.26. ↩

  12. Fabian Trautsch, "An Analysis of the Differences between Unit and Integration Tests", PhD thesis, University of Göttingen, 2019, doi:10.53846/goediss-7393. ↩

  13. Fabian Trautsch, Steffen Herbold, Jens Grabowski, "Are Unit and Integration Test Definitions Still Valid for Modern Java Projects? An Empirical Study on Open-Source Projects", Journal of Systems and Software 159, 2020, doi:10.1016/j.jss.2019.110421. ↩