Zero-Surface Refactors: Fingerprint the Public Interface Before You Split the Diff

Characterization tests pin behavior. They say nothing about the interface. A refactor can keep every test green and still rename a public function, drop a default argument, or change a class base.

So treat the public surface as a hard constraint. Measure it mechanically before and after. Split the diff the moment it moves.

The rule for this walkthrough: a refactor commit must leave the exported surface byte-identical. Behavior pins tell you the code still works. Surface pins tell you the contract still exists.

Why behavior tests miss interface drift

A golden master captures one call path. Your consumers call other paths. Rename parse_report to parse_report_doc and the golden master still passes, because the test imports the old name from the module directly.

Defaults are worse. Changing def build(path, strict=False) to def build(path, strict=True) changes nothing in your test suite. It changes behavior for every caller that never passed that flag.

So add a second, cheaper gate. It parses source, not runtime. It costs milliseconds and catches the drift your tests structurally cannot see.

Step 1 — Write a deterministic surface fingerprint

This script walks a package, parses each file with ast, and records every public top-level name with its kind and signature. Output is sorted JSON, so two runs of unchanged code are identical.

#!/usr/bin/env python3 """surface.py - deterministic public-surface fingerprint for a Python package. Usage: python surface.py --root src --package mypkg > .surface/before.json python surface.py --root src --package mypkg > .surface/after.json python surface.py --diff .surface/before.json .surface/after.json """ import argparse import ast import json import pathlib import sys KINDS = (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef, ast.Assign, ast.AnnAssign) def _public(name): return bool(name) and not name.startswith("_") def _sig(fn): a = fn.args parts = [] for arg in list(getattr(a, "posonlyargs", [])) + list(a.args): parts.append(arg.arg) if a.vararg: parts.append("*" + a.vararg.arg) elif a.kwonlyargs: parts.append("*") for arg, default in zip(a.kwonlyargs, a.kw_defaults): # default is None means "no default"; ast.Constant(None) is a real default  parts.append(arg.arg if default is None else arg.arg + "=") if a.kwarg: parts.append("**" + a.kwarg.arg) return "(" + ",".join(parts) + ")" def _bases(node): names = [] for base in node.bases: names.append(getattr(base, "id", getattr(base, "attr", "?"))) return ",".join(names) def _entry(node): if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)): return "def" + _sig(node) if isinstance(node, ast.ClassDef): return "class(" + _bases(node) + ")" return "assign" def _assigned(node): targets = node.targets if isinstance(node, ast.Assign) else [node.target] names = [] for target in targets: for sub in ast.walk(target): if isinstance(sub, ast.Name): names.append(sub.id) return names def fingerprint(root, package): out = {} for path in sorted(pathlib.Path(root).rglob("*.py")): rel = path.relative_to(root).with_suffix("") parts = list(rel.parts) if parts and parts[-1] == "__init__": parts = parts[:-1] dotted = ".".join(([package] if package else []) + parts) tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path)) for node in tree.body: if not isinstance(node, KINDS): continue if isinstance(node, (ast.Assign, ast.AnnAssign)): for name in _assigned(node): if _public(name): out[dotted + ":" + name] = "assign" continue if _public(node.name): out[dotted + ":" + node.name] = _entry(node) return out def report(before, after): added = sorted(set(after) - set(before)) removed = sorted(set(before) - set(after)) changed = sorted(k for k in set(before) & set(after) if before[k] != after[k]) for label, keys in (("ADDED", added), ("REMOVED", removed), ("CHANGED", changed)): for key in keys: old = before.get(key, "-") new = after.get(key, "-") print("%s %s %s -> %s" % (label, key, old, new)) print("delta=%d added=%d removed=%d changed=%d" % ( len(added) + len(removed) + len(changed), len(added), len(removed), len(changed))) return 1 if (added or removed or changed) else 0 def main(argv): p = argparse.ArgumentParser() p.add_argument("--root", default="src") p.add_argument("--package", default="") p.add_argument("--diff", nargs=2, metavar=("BEFORE", "AFTER")) args = p.parse_args(argv) if args.diff: before = json.loads(pathlib.Path(args.diff[0]).read_text(encoding="utf-8")) after = json.loads(pathlib.Path(args.diff[1]).read_text(encoding="utf-8")) return report(before, after) print(json.dumps(fingerprint(args.root, args.package), indent=2, sort_keys=True)) return 0 if __name__ == "__main__": sys.exit(main(sys.argv[1:])) 

Run it twice without editing anything. The two JSON files must be identical. If they differ, your parser is not deterministic yet, and the gate is worthless.

python surface.py --root src --package mypkg > /tmp/a.json python surface.py --root src --package mypkg > /tmp/b.json cmp /tmp/a.json /tmp/b.json && echo "stable" 

Step 2 — Snapshot the surface before you touch a line

Commit or stash first. Then write the baseline into the repo, so the artifact travels with the branch.

mkdir -p .surface python surface.py --root src --package mypkg > .surface/before.json git add .surface/before.json && git commit -m "chore: pin public surface" 

Do not regenerate before.json later. Its whole value is that it was written by code you have not edited yet.

Step 3 — Freeze behavior with one probe per public name

Surface pins prove a name exists. They do not prove the name still works. Generate probes from the baseline file so coverage is mechanical, not accidental.

# tests/test_surface.py import importlib import json import pathlib import pytest SURFACE = json.loads(pathlib.Path(".surface/before.json").read_text(encoding="utf-8")) @pytest.mark.parametrize("key", sorted(SURFACE)) def test_public_name_still_resolves(key): module_name, attr = key.split(":", 1) module = importlib.import_module(module_name) assert hasattr(module, attr), "%s vanished from %s" % (attr, module_name) 

This is a smoke test, not a characterization test. Add one real call per name you are about to touch. Import costs nothing. A wrong default costs a production incident.

Step 4 — Make the smallest change, then re-diff

Change one thing. Re-run both gates. Then read the verdict, and do not negotiate with it.

python surface.py --root src --package mypkg > .surface/after.json python surface.py --diff .surface/before.json .surface/after.json; echo "exit=$?" pytest -q tests/test_surface.py 

Exit code 1 is the gate firing. Never pipe it into true. A gate you silence is a comment.

The split rule that keeps diffs small

When the delta is non-zero, do not rewrite the refactor. Move the interface change out.

1. Revert the hunks that touch public names or signatures.
2. Land the pure-move commit with delta 0, reviewed as a refactor.
3. Land the API change separately, with a changelog entry and a version bump.

This is the practical meaning of "smallest safe change". Safe is not a feeling about the diff size. Safe is a measured property of the commit.

Where free model access and the free server fit

Writing one probe per public name is mechanical, and mechanical work is a good fit for a model. I used MonkeyCode's free model access to draft the probe bodies from the keys in before.json, then deleted every probe that asserted a value I had not verified by reading the source. The model proposed the tests. It did not decide what was true.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The free server option is where the gate can live as a small always-on job instead of a laptop ritual. Run surface.py --diff plus pytest -q there before a merge, and keep the two JSON files as build artifacts. Treat that host as a convenience tier, not a production guarantee. Check the current terms yourself before you put anything sensitive on it.

If the free tier disappears tomorrow, the workflow survives. The script is 90 lines of stdlib Python and it does not import the product.

What the fingerprint cannot see

Be explicit about the blind spots, or you will trust it too much.

• Dynamic exports. globals().update(...) and module-level __getattr__ produce names the AST never lists.
• Decorators. The recorded signature is the undecorated one, so a wrapper that changes arity is invisible.
• Star imports. from .core import * re-exports nothing this script can name.
• Runtime behavior. Defaults are recorded as present or absent, never as correct.
• Cross-version drift. Run the script under the interpreter you ship on, not an older one lying around.

Who should not use this

The rule is wrong for libraries whose API is legitimately in flux, and for pre-1.0 packages where renaming is the point. It is useless without a runnable test suite, because delta 0 plus zero tests proves nothing. It is also a poor fit for codebases that export through a registry or plugin loader rather than module attributes.

For everyone else, it is the cheapest abuse test you can run on a refactor. Start with the stability check in Step 1. It needs no account and no network, and it fails loudly on the first real drift.