Swap One Seam, Dual-Run Both Implementations, Then Delete the Old Path

The smallest safe refactor is one seam swap. Not a module rewrite. Swap the call site, run both implementations on the same inputs, compare, then delete.

This post covers three parts. A dual-run harness. A gate that fails on unlabeled mismatches. A triage step for the ledger.

Core rule: a mismatch you have not explained is a behavior change. Treat it as one until a reviewer labels it.

Why a Pinned Net Stops Here

Golden masters freeze the inputs you chose to record. They say nothing about traffic you never captured. Dual-run samples the live distribution instead of your imagination.

Both nets are useful. They cover different failure modes. Pin bytes for known regressions, dual-run for the cases you forgot.

Step 1 — Pick the Seam by Call-Site Count

A seam is a function boundary, not a package. Small blast radius beats clean architecture here.

Count before you choose:

git grep -n 'parse_report(' -- '*.py' | wc -l git grep -ln 'parse_report(' -- '*.py' 

Keep one seam per change. Keep call sites at five or fewer. Above that, insert a wrapper first and swap the wrapper later.

Write down three facts before touching code. Return type, error contract, and side effects. If the function writes files or calls the network, you are not dual-running yet. You are sequencing.

Step 2 — Record Inputs at the Seam

A decorator probe gives you real arguments with no test authoring. Cheap and boring, which is the point.

import json, functools, pathlib CAPTURE = pathlib.Path('seam-inputs.jsonl') def record(fn): @functools.wraps(fn) def wrapper(arg): with CAPTURE.open('a', encoding='utf-8') as fh: fh.write(json.dumps({'fn': fn.__name__, 'arg': arg}, default=str) + '\n') return fn(arg) return wrapper 

This writes raw arguments. Redact before the file leaves the box. Drop secrets, tokens and identifiers you cannot store.

One day of capture is usually enough for a first pass. Add hand-written edge cases to the same file. Empty input, huge input, and one malformed record.

Step 3 — Dual-Run Both Implementations

The harness runs both functions on each case. It fingerprints normalized output and records errors separately.

import json, time, hashlib, pathlib from dataclasses import dataclass, asdict LEDGER = pathlib.Path('mismatch-ledger.jsonl') def norm(value): if isinstance(value, dict): return {k: norm(v) for k, v in sorted(value.items())} if isinstance(value, list): return [norm(v) for v in value] if isinstance(value, float): return round(value, 6) return value def fingerprint(payload): blob = json.dumps(payload, sort_keys=True, default=str).encode() return hashlib.sha256(blob).hexdigest()[:12] @dataclass class Row: input_id: str old: str new: str equal: bool old_error: str | None new_error: str | None def run_one(fn, arg): start = time.monotonic() try: out = fn(arg) except Exception as exc: return None, f'{type(exc).__name__}: {exc}', time.monotonic() - start return out, None, time.monotonic() - start def dual_run(old_fn, new_fn, cases, log=LEDGER): with log.open('a', encoding='utf-8') as fh: for case in cases: old_out, old_err, old_ms = run_one(old_fn, case['arg']) new_out, new_err, new_ms = run_one(new_fn, case['arg']) row = Row( input_id=case['id'], old='' if old_err else fingerprint(norm(old_out)), new='' if new_err else fingerprint(norm(new_out)), equal=(old_err is None and new_err is None and norm(old_out) == norm(new_out)), old_error=old_err, new_error=new_err, ) fh.write(json.dumps({**asdict(row), 'old_ms': round(old_ms, 3), 'new_ms': round(new_ms, 3)}) + '\n') 

Two details matter. Errors are data, not crashes. Latency lands in the same row, so a slower swap is visible immediately.

Step 4 — Normalize Before You Compare

Every normalizer is a claim about semantics. Write the claim down or delete the normalizer.

Start with zero normalizers. Add one only when you can name the caller expectation it preserves.

Step 5 — Triage the Ledger Instead of Skimming It

Most mismatches are formatting noise. A few are real. Reading hundreds of rows by hand gets sloppy around row forty.

This is where a model helps. MonkeyCode provides free model access and a free server option (operator-supplied availability), so a short labeling pass costs you no budget decision.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Use it as a classifier, not an approver. Three labels, no prose.

# Illustrative pseudocode: the call shape depends on your client. LABELS = ['same_semantics', 'real_change', 'unknown'] prompt = f'''You label one diff. Answer with one label only. Old output: {old_out} New output: {new_out} Labels: {LABELS} Label:''' 

Append the returned label to the ledger row. Nothing else. A human confirms every real_change.

Keep payloads clean. Send redacted or synthetic rows only. If your data cannot leave the network, run the model locally or skip this step entirely.

Step 6 — Gate on Unlabeled Rows

The gate fails for two reasons. Unlabeled rows, or unresolved real changes. Exit code carries the verdict into CI.

import json, collections, sys tally = collections.Counter() for line in open('mismatch-ledger.jsonl', encoding='utf-8'): row = json.loads(line) if row['equal']: tally['match'] += 1 else: tally[row.get('label') or 'unlabeled'] += 1 print(dict(tally)) sys.exit(0 if tally['unlabeled'] == 0 and tally['real_change'] == 0 else 1) 

Run it twice on two input sets. Recorded traffic first. Edge cases second. Both must reach zero unresolved rows before you delete the old function.

When Dual-Run Is the Wrong Tool

Limits and Who Should Skip This

Dual-run detects divergence on sampled inputs. It never proves correctness. Untested branches stay untested.

Latency doubles during the run. Sample five to ten percent on busy paths. Do not shadow a payment write.

Skip this approach in three cases. You cannot legally store the recorded inputs. The function is a thin wrapper over two lines. Your team has no reviewer for the ledger.

Also skip it when the old implementation is already deleted. Then you have the other problem.

If you already have a seam you are afraid to swap, record its inputs for a day and run the harness above. The first ledger is usually more interesting than the diff.