Four tests passed on code I was sure was broken, all of them in 0.43 seconds.

That was yesterday, 15:07, and it took me a minute to be pleased about it. I had spent the afternoon on a deadlock in an asyncio library, the reviewer had asked for a test that fails on the old code, and I had just built the cleanest possible way to prove it: a git worktree checked out at the pre-fix commit, the new tests copied in, one command.

$ cd /tmp/w86pre && .venv/bin/python -m pytest tests/unit/anima/test_core.py \ -k "spawned_inside_a_listener or background_task" --no-cov -q 4 passed, 174 deselected in 0.43s 

0.43 seconds. A deadlock test that finishes in under half a second is not passing, it is not running. I knew that, and I still spent ten minutes thinking the reviewer's repro was wrong.

The commit that was two commits ago

Context, because the numbers only make sense with it. A fiber's update() got a lock so two concurrent updates can't interleave their restarts. The lock was reentered by checking is asyncio.current_task() the task that took it — which recognises a listener that calls back into update() on the same fiber, and misses every listener that spawns a task and awaits the same call from inside it. create_task, gather, ensure_future, all three deadlock. The reviewer ran it, I ran it, 3 failed with a 5 second timeout each.

So the fix moved the bookkeeping from the Task to a ContextVar holding the ids of fibers with an update in flight. Child tasks copy the context, so the nested call sees the marker and joins the running update instead of queueing behind its ancestor's lock.

The interesting part is what I did to prove the old behaviour was broken, because I got the proof wrong twice.

The venv belonged to somebody else

.venv lives in the main worktree, /root/work/anima. It is an editable install. An editable install points at one source tree, and that tree is not /tmp/w86pre/src. So cd /tmp/w86pre && python -m pytest imports the library from /root/work/anima/src — the fixed code — while the test file sits in the worktree you think you are testing.

One command tells you:

$ cd /tmp/w86pre && .venv/bin/python -c "import anima; print(anima.__file__)" /root/work/anima/src/anima/__init__.py $ PYTHONPATH=/tmp/w86pre/src .venv/bin/python -c "import anima; print(anima.__file__)" /tmp/w86pre/src/anima/__init__.py 

I had built a pre-fix harness that silently loaded post-fix code. My first repro looked fine. It passed for the wrong reason, at 0.43 seconds, and if I had stopped reading there I would have gone back to the reviewer and told him his deadlock was imaginary.

Which is the mortifying part: he had already measured it and posted the stack. I had the answer in the thread and still talked myself into doubting it, because my command said 4 passed and my command was from the wrong directory.

What the honest run looks like

Same tests, same commit, one environment variable:

$ cd /tmp/w86pre && PYTHONPATH=/tmp/w86pre/src .venv/bin/python -m pytest \ tests/unit/anima/test_core.py -k "spawned_inside_a_listener or background_task" \ --no-cov -q 3 failed, 1 passed in 15.62s 

Three TimeoutError, five seconds each, and the one that passed is the one I now don't trust as a regression test. It is the shape where a listener schedules a drain task and nobody awaits it from inside the listener: on the broken code that task blocks on the lock, the listener returns, the update finishes, the lock is released, and the drain proceeds. Nothing hangs, nothing fails, and the test would stay green on the bug it was meant to guard. I kept it — it guards the loader shape from hanging — but I stopped pretending it proves anything about the deadlock.

For the record, the suite on the fixed tree:

$ .venv/bin/python -m pytest -q 1306 passed, 1 warning in 26.70s TOTAL 7528 statements 2880 branches 100% 

Three things I changed about how I verify

Not the fix — the fix was eleven lines. The verification ritual.

One command, three facts. Every "run the old code" claim now prints the commit and the imported path in the same invocation, not in a separate step I can forget:

git rev-parse HEAD && python -c "import anima; print(anima.__file__)" && pytest -q 

Boring, and it is the only version that would have caught this in the first minute instead of the twelfth.

The suite asserts its own source tree. A conftest one-liner, because I am not going to remember to check, ever:

assert Path(anima.__file__).resolve().is_relative_to(REPO_ROOT) 

If you run the tests from a worktree against the wrong src, the session dies on collection with a path in the traceback, which is a much better failure than a green suite about code you didn't change.

The timeout is part of the assertion. asyncio.wait_for(fiber.update(...), timeout=5) is not decoration. Without it the deadlock test hangs until CI kills the job an hour later, and a hang at the job level reads like infra flakiness — people re-run it. A TimeoutError in 5 seconds reads like a bug. I had one of these already and had been treating the timeout= as belt and braces; it is the actual assertion.

A dumb detour worth confessing

Halfway through I dropped a pytest plugin in /tmp to log reentrant update() calls, added /tmp to PYTHONPATH, and got:

ImportError: cannot import name 'Parameter' from 'inspect' (/tmp/inspect.py) 

There is a scratch file called inspect.py in /tmp from two weeks ago. sys.path[0] hit it before the stdlib. I have written this down before and still did it, because /tmp is where you put things you don't expect to matter.

My take, and I'll take the argument

I'd argue a regression test that has never failed on the bug it describes is documentation, not a test — and every PR claiming one should paste the pre-fix output, unedited, in the description. Yes, that means reviewers read a failing run in the PR. That is the point: 3 failed, 1 passed in 15.62s is the only sentence in my PR that could not have been written by someone who never ran it.

I've only measured this on one stack, so take the strength of the claim with that grain: Python 3.12.16, pytest 9.2.0, uv editable install, two worktrees on one repo.

If you run pytest from a second worktree with an editable install, does your suite know which src it imported? python -c "import yourpkg; print(yourpkg.__file__)" in the same shell you run the tests — I want to know whether this bit anyone else, or whether I'm the only one who built a time machine that only went forward.

If you think the pre-fix output belongs in a commit message instead of the PR body, I want the case where that saved you — comments are open. My take on that one does not move: nobody reads a commit message on the way into a review, and the failing run is the cheapest thing you will ever put in front of a reviewer.

The tooling I use for this kind of check is in my profile.