Cap the Context, Not the Review: A Byte-Budgeted Evidence Packet for OSS Patches

A maintainer opens a pull request that touches eleven files and rewrites one hot path in a parser. The diff is roughly nine hundred lines, and the accompanying test run fails in two places that look unrelated to the stated goal. A model is handed the entire diff plus three full source files, and it returns thirty review comments within seconds. Three of those comments survive contact with the actual code, and the maintainer spends more time discarding noise than reading the diff would have cost.

The problem in that composite scenario is not model quality. It is that the review request carried unlimited context and zero verifiable claims, so every confident sentence looked equally credible. The workflow below treats a model review as a bounded, auditable artifact rather than a conversation, and it spends its cheapest compute on the two stages where mistakes are least expensive: reproduction and first-pass triage.

The two free stages in this workflow use MonkeyCode's free model access and free server option, both supplied by the project's operator for this write-up. MonkeyCode is described as an open-source coding agent project, and the operator reports a free allowance of roughly ten million tokens alongside the free server. Treat every availability number as time-sensitive and confirm it on the project's own page before planning around it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The scripts below are illustrative and were not executed against a specific upstream repository for this article. They are written to be adapted, and each one states its assumptions inline.

The failure mode: a context dump instead of evidence

A context dump review fails in three predictable ways, and naming them makes the fix obvious. Line numbers drift because the model sees a diff without a stable head revision. Style opinions crowd out behavioral findings because nothing in the request distinguishes severity. Claims arrive without a command, so verifying one costs the same as finding it yourself.

Bounding the input and forcing execution-ready claims fixes all three at once. The packet carries the head commit, the exact test command, and the failing test names, which makes every later claim checkable against a known revision. A byte cap keeps the cheap model cheap and, more importantly, keeps the review focused on the code that the failure actually touches.

Step 1: Reproduce on a machine you can throw away

Reproduction belongs on a disposable box, because a local run inherits every cached dependency and every half-finished branch on the developer's laptop. A free server is a reasonable fit here precisely because the work is boring, short-lived, and destructive: clone, check out a commit, run one command, record the result, discard the machine. It should never hold credentials, unpublished code, or anything that must survive the session.

The script below writes a receipt that later steps consume, and it fails loudly when the working tree is dirty. A dirty tree is the most common reason a receipt cannot be reproduced by a second person.

#!/usr/bin/env bash # receipt.sh — fingerprint one patch attempt. Illustrative; adapt paths first. set -uo pipefail REPO="${1:?usage: receipt.sh <repo-dir> <test-command>}" CMD="${2:?usage: receipt.sh <repo-dir> <test-command>}" cd "$REPO" || exit 2 SHA="$(git rev-parse HEAD)" BRANCH="$(git rev-parse --abbrev-ref HEAD)" DIRTY="$(git status --porcelain | wc -l | tr -d ' ')" { python3 --version 2>&1 || echo python=missing node --version 2>&1 || echo node=missing go version 2>&1 || echo go=missing } > /tmp/toolchain.txt LOG=/tmp/test.log bash -c "$CMD" > "$LOG" 2>&1 STATUS=$? # Only failing identifiers travel downstream; full logs stay local. grep -Eo '(FAILED|not ok)[^[:cntrl:]]{0,120}' "$LOG" | head -n 40 > /tmp/failures.txt python3 - "$SHA" "$BRANCH" "$DIRTY" "$CMD" "$STATUS" <<'PY' > receipt.json import json, sys, datetime sha, branch, dirty, cmd, status = sys.argv[1:6] rd = lambda p: open(p).read().splitlines() print(json.dumps({ "commit": sha, "branch": branch, "dirty_files": int(dirty), "command": cmd, "exit_code": int(status), "toolchain": rd("/tmp/toolchain.txt"), "failures": rd("/tmp/failures.txt"), "captured_at": datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ"), }, indent=2)) PY echo "wrote receipt.json (exit=$STATUS)" [ "$DIRTY" -eq 0 ] || echo "warning: dirty tree, receipt is not reproducible" 

Run it against the base commit first and against the patch head second. Two receipts let a reviewer see whether the failure predates the patch, which is the single most useful fact in triage and the cheapest one to obtain.

Step 2: Cap the packet before the model sees it

The packet is the only thing the model receives, so it should contain the failure, the touched hunks, and nothing decorative. Ranking changed files by whether a failing test mentions their path pushes the relevant hunks to the front of a hard byte budget. A per-file cap stops one generated file from consuming the entire allowance.

#!/usr/bin/env python3 """packet.py — assemble a byte-capped evidence packet. Illustrative.""" import argparse, json, subprocess def run(*args): return subprocess.run(args, capture_output=True, text=True).stdout def main(): ap = argparse.ArgumentParser() ap.add_argument("--base", required=True) ap.add_argument("--head", default="HEAD") ap.add_argument("--receipt", default="receipt.json") ap.add_argument("--budget", type=int, default=24000) ap.add_argument("--out", default="packet.md") a = ap.parse_args() receipt = json.load(open(a.receipt)) files = run("git", "diff", "--name-only", f"{a.base}...{a.head}").split() blob = " ".join(receipt.get("failures", [])) files.sort(key=lambda f: f not in blob) # failure-adjacent files first  header = ["# Evidence packet", "", f"- commit: {receipt['commit']}", f"- command: `{receipt['command']}`", f"- exit: {receipt['exit_code']}", f"- failures: {len(receipt.get('failures', []))}", ""] budget = a.budget - len("\n".join(header).encode()) body, omitted = [], 0 for f in files: if budget <= 0: omitted += 1 continue hunk = run("git", "diff", "-U3", f"{a.base}...{a.head}", "--", f) hunk = hunk[:min(6000, budget)] budget -= len(hunk.encode()) body.append(hunk) text = "\n".join(header + body) if omitted: text += f"\n[truncated: {omitted} files omitted by byte budget]\n" open(a.out, "w").write(text) print(f"{a.out}: {len(text.encode())} bytes, {len(files)} changed files") if __name__ == "__main__": main() 

A budget near twenty-four kilobytes is a working default rather than a rule, and it fits comfortably inside the reported free allowance for a long triage session. The cap is also a truthfulness device: when the packet says files were omitted, the reviewer knows the model did not see them and stops trusting a verdict about the untouched parts of the repository.

Step 3: Replace prompt wishes with a review contract

An instruction like "find bugs and be thorough" produces prose. A contract produces records that a script can accept or reject before a human reads anything. The contract below has four required fields and one hard rule: no claim without a command that would demonstrate it.

You are reviewing a patch from an evidence packet. Output JSON Lines only. Each line: {"claim": str, "file": str, "line": int, "command": str, "expected": str} Rules: 1. file must appear in the packet; line must be a real line number in that file. 2. command must be a single test command drawn from the packet's test suite. 3. expected must describe the observable result that would prove the claim. 4. If a concern cannot be tied to a command, omit it. 5. Maximum 12 lines. Rank behavioral risk above style. 

The filter that follows is deliberately mechanical, and it is where most free-model output is discarded without argument. Claims pointing at nonexistent files, out-of-range lines, or non-allowlisted commands never reach a human, which keeps the reviewer's attention on the claims that could actually change the patch.

#!/usr/bin/env python3 """check_claims.py — reject unverifiable claims. Illustrative.""" import json, os, shlex, sys ALLOWED = {"pytest", "go", "npm", "cargo", "make", "tox"} def main(path, root): kept = dropped = 0 for n, raw in enumerate(open(path, errors="replace"), 1): try: c = json.loads(raw) except json.JSONDecodeError: print(f"DROP line {n}: not json", file=sys.stderr); dropped += 1; continue target = os.path.join(root, str(c.get("file", ""))) if not os.path.isfile(target): print(f"DROP line {n}: file missing", file=sys.stderr); dropped += 1; continue lines = sum(1 for _ in open(target, errors="replace")) if not 1 <= int(c.get("line", 0)) <= lines: print(f"DROP line {n}: line out of range", file=sys.stderr); dropped += 1; continue parts = shlex.split(c.get("command", "").strip() or "") if not parts or parts[0] not in ALLOWED: print(f"DROP line {n}: command not allowlisted", file=sys.stderr); dropped += 1; continue kept += 1; print("KEEP", json.dumps(c)) print(f"{kept} kept / {kept + dropped} claims", file=sys.stderr) if __name__ == "__main__": main(sys.argv[1], sys.argv[2]) 

Run the surviving commands before reading their prose explanations. A claim whose command passes has been falsified by execution, and a claim whose command fails now has a failing test that can become part of the contributor's next commit.

Step 4: Triage what survives the checks

Classification is a human job, and the table below keeps it short enough to apply consistently across a review queue. The middle column names who decides, and the right column names the evidence that must exist before the decision is recorded.

The table is valuable mostly for what it excludes. A free model can flood a queue with plausible style remarks, and routing those to the contributor with a linter reference removes them from maintainer attention without losing them. Security and performance stay with a human because both require judgment about threat models and acceptable trade-offs that no review packet contains.

Where free models actually earn their place

Free access changes the economics of the boring stages, not the judgment stages. Summarizing a long failure log into a handful of distinct error signatures is cheap, reversible, and easy to verify against the log itself. Spotting that a changed conditional has no corresponding test case is a pattern-matching task with a mechanical check. Finding dead code, inconsistent naming, and comments that contradict the diff are all low-risk observations whose worst outcome is a rejected claim.

The same access is a poor fit for intent questions. Whether a patch should change behavior, whether a public interface can break, and whether a test encodes a requirement or an accident are decisions that depend on project history the packet never contains. Attempting to delegate those makes a fast triage loop feel authoritative while quietly moving risk into the merge.

Limitations, and who should skip this

• The byte budget creates false negatives. A real problem in an omitted file will not be found, and the packet must state what it dropped.
• Free tiers carry rate limits, queueing, and terms that change without notice. A workflow that cannot tolerate a slow response is a bad fit.
• Never send unpublished or proprietary code to a free endpoint. This workflow assumes a public fork and public test output.
• Free servers are convenient and untrusted. Put nothing durable on them, and prefer short sessions over long-lived state.
• The scripts here are starters. Repositories with generated files, binary fixtures, or multi-language monorepos need their own ranking rules.
• A reviewer who reads the full diff before trusting any packet conclusion still catches what the packet hid, so the cap is a filter and not a replacement.

Maintainers of small projects with slow test suites get the most from the reproduction stage, because a disposable machine with a clean receipt answers the "does it even fail" question in one command. Projects with security-sensitive patch surfaces should keep the packet out of the loop entirely and rely on human review with a reproducible harness.

A closing note on cost discipline

A bounded packet, a recorded receipt, and an executable claim format turn a free model into a predictable instrument rather than an opinion generator. The budgeted stages handle volume, the filter discards what cannot be verified, and the maintainer spends attention only on claims that survived execution. If a maintainer wants to try the two cheap stages without standing up local infrastructure, MonkeyCode's free server option is the least-friction starting point, and the operator's reported free token allowance makes the packet approach affordable for a trial period.

The durable lesson is independent of any tool. A patch review should carry its evidence with it, and any reviewer, human or otherwise, should be asked for claims that a command can settle.