Output equality is not behavior equality. A refactor can return the same value and still reorder two money calls. If your baseline only pins return values, that change ships green.
So pin the interactions before you touch the seam. Record what the function calls, in what order, with which arguments. Then make one small change. Then diff the log.
Everything below is illustrative. The code is unexecuted. Treat it as a plan you adapt, not a result you inherit.
What to pin, and what to normalize
Not every observation deserves a snapshot. Pin contracts. Normalize noise.
| Observation | Pin it? | Reason |
|---|---|---|
| Return value | Yes | Cheapest public contract |
| Order of side-effect calls | Yes | Reordering charges and receipts is a real bug |
| Argument values at the call | Yes | Catches silent amount-math drift |
| Call counts | Yes | Duplicate charges hide in loops |
| Log lines | Only if ops depends on them | Noisy and rarely contractual |
| Timestamps, UUIDs, row ids | Normalize, do not pin | Nondeterministic across runs |
| Internal helper calls | No | You want freedom to reshape privates |
That last row matters. Pin too deep and you cannot refactor at all. Pin too shallow and the harness says nothing.
Prerequisites
- A function with outbound calls you cannot easily observe.
- Python 3.8+ or an equivalent spy mechanism in your language.
- A test runner that writes a snapshot to disk, not just a pass/fail line.
- Agreement on which collaborators are contractual.
Step 1: Write a spy that records calls
No third-party dependency needed. Wrap the collaborator, record the call, delegate.
# illustrative: snapshot_spy.py import functools class Spy: def __init__(self): self.calls = [] def wrap(self, label, target): @functools.wraps(target) def inner(*args, **kwargs): self.calls.append( {"label": label, "args": list(args), "kwargs": kwargs} ) return target(*args, **kwargs) return inner The spy records before delegating. If the real call raises, you still know it was attempted. That distinction will save you later.
Step 2: Capture a baseline once
Here is the messy code under test. Note it calls two module-level collaborators directly.
# illustrative: renewals.py import billing def renew(subscription): if subscription["status"] != "active": return "skipped" amount = subscription["plan_price"] - subscription.get("credit", 0) if amount < 0: amount = 0 billing.charge_card(subscription["card"], amount) billing.send_receipt(subscription["email"], amount) return "charged" Now the capture script. It swaps the module attributes, runs two fixture cases, and writes JSON.
# illustrative: capture_baseline.py import json import billing import renewals from snapshot_spy import Spy CASES = [ {"status": "active", "plan_price": 2000, "credit": 500, "card": "card_A", "email": "a@example.test"}, {"status": "active", "plan_price": 1000, "credit": 5000, "card": "card_B", "email": "b@example.test"}, ] def capture(): spy, results = Spy(), [] billing.charge_card = spy.wrap("charge_card", lambda card, amount: {"ok": True}) billing.send_receipt = spy.wrap("send_receipt", lambda email, amount: None) for case in CASES: outcome = renewals.renew(dict(case)) results.append({"case": case, "returned": outcome, "calls": spy.calls}) spy.calls.clear() return results if __name__ == "__main__": with open("baseline.json", "w") as fh: json.dump(capture(), fh, indent=2, sort_keys=True) # illustrative, not executed python capture_baseline.py diff baseline.json baseline.previous.json That diff is the whole discipline. Empty diff means the interaction shape held.
Two behaviors worth naming now. The second fixture is the floor case: 1000 - 5000 clamps to 0. Your snapshot should show 0 in the argument list, not -4000. If it shows the negative number, you found a bug before refactoring.
Step 3: Normalize before you trust the baseline
Run the capture twice into two files. Diff them. Any field that moved is nondeterministic and must become a placeholder.
# illustrative: normalize.py import re def scrub(value): if isinstance(value, str): return re.sub(r"[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}", "<uuid>", value) if isinstance(value, dict): return {k: scrub(v) for k, v in value.items()} if isinstance(value, list): return [scrub(v) for v in value] return value A stable baseline is the only kind worth committing. A flaky one trains you to rerun until green.
Step 4: Make the smallest safe change
Only one edit. Give the function a gateway parameter that defaults to the module object.
# illustrative: renewals.py after the single edit import billing def renew(subscription, gateway=billing): if subscription["status"] != "active": return "skipped" amount = subscription["plan_price"] - subscription.get("credit", 0) if amount < 0: amount = 0 gateway.charge_card(subscription["card"], amount) gateway.send_receipt(subscription["email"], amount) return "charged" Why bind to the module object and not to a call like billing.charge_card? Because the default captures the module, so later patching of billing.charge_card still reaches through. Binding the function object would freeze today's reference and quietly break existing tests. That is the kind of subtle difference an interaction snapshot can catch and a return-value test cannot.
Order is unchanged. Argument computation is unchanged. The call log should be byte-identical.
Step 5: Verify, and prove the harness bites
# illustrative, not executed python capture_baseline.py diff baseline.json baseline.previous.json Empty diff is necessary but not sufficient. Confirm the harness can fail. In a scratch branch, swap the two gateway calls and rerun. The diff must show both entries moved. If it stays empty, your spy is not wired to the code path and the whole baseline is worthless.
Checklist before merging:
- Baseline diff is empty.
- Deliberate call-order swap produced a visible diff.
- Only one production line group changed.
- The floor case still logs
0.
Where a free model helps, honestly
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The slow part of this workflow is not the harness. It is reading a large module and listing every outbound collaborator per function. That is a good fit for free model access: paste one file, ask for a table of function name, collaborator, and whether the call is contractual. You review every row; the model's list is a draft, not an authority. The free server option is a reasonable place to run step 2 off your laptop when the repo is heavy, but check the product's current limits and terms yourself before putting real code there. My framing here rests on the operator-stated availability of free models and a free server, not on measured quotas. If the model hallucinates a collaborator, your diff against the real baseline catches it in seconds. That gate is the point: the model proposes, the snapshot disposes.
Who should skip this
- Code with no outbound calls. Pure functions need output tests, not spies.
- Teams without a stable fixture format. Normalization debt will drown you.
- Hot paths where the spy wrapper distorts timing you are measuring.
- Anyone who will pin private helpers and then complain they cannot refactor.
The narrow claim stands: output equality under-tests interaction order. Pin the log, make one edit, diff the log. If you want a starting point for the seam inventory, hand the free model one file and keep your review pen ready.
