The Restore Take-Home: Grade Whether a Candidate Can Rebuild an Agent Environment From a README Alone

Most take-homes grade the artifact a candidate ships, which teaches the reviewer almost nothing about how that candidate handles an environment they did not build. The restore take-home inverts the exercise: the candidate receives a README, a half-broken setup, and no original author to ask. What gets scored is not the agent itself but a receipt proving that someone else can rebuild the same working agent from cold. Below are the prompt, a weighted rubric, a reference solution shape, and the failure modes that keep recurring in review.

Why README-only restore separates candidates

The task is deliberately unglamorous, because environment reasoning is where most agent projects actually fail in production. A candidate who can read a stale README, notice what no longer holds, and document the delta is far more valuable than one who ships an impressive demo tied to their laptop. The exercise also produces comparable evidence: every submission contains commands, timings, and classifications that a reviewer can re-run.

• The reviewer can replay the receipt instead of trusting a screen recording.
• Ambiguity becomes visible, because the README is intentionally incomplete.
• Cost and rate-limit behavior gets exercised on a free tier, where boundaries appear quickly.
• The candidate must decide what to change versus what to preserve in someone else's setup.

The prompt

Hand the following to the candidate verbatim, along with a repository that contains a stale README, a requirements.txt, and a client that calls an OpenAI-compatible endpoint.

You have 3 hours. The author of this repository is unavailable. 1. Restore a working environment from README.md alone. Where the README is wrong or silent, decide, then record the decision and your reason. 2. Produce receipt.tsv with one line per state-changing command you ran, in order, with a UTC timestamp. 3. Produce TRIAGE.md that classifies every failed probe you hit, using the categories auth, rate_limit, cold_start, network, contract_mismatch. 4. Produce TEARDOWN.md describing how to remove everything you created. 5. Do not edit the application source. You may add scripts and docs. 

The rubric

Scores should be assigned per dimension, not holistically, so two reviewers reach similar conclusions on the same submission.

Reference solution shape

The snippets below are a rubric anchor rather than a tested release; they show the level of rigor a full-credit submission reaches. A passing submission typically contains more comments and more recorded decisions than code.

setup.sh

#!/usr/bin/env bash set -euo pipefail STATE_DIR="${STATE_DIR:-$HOME/.restore-task}" mkdir -p "$STATE_DIR" record() { printf '%s\t%s\n' "$(date -u +%FT%TZ)" "$1" >> "$STATE_DIR/receipt.tsv"; } command -v python3 >/dev/null || { echo "missing python3"; exit 1; } record "found_python $(python3 -V 2>&1)" python3 -m venv "$STATE_DIR/venv" 2>/dev/null || true # shellcheck disable=SC1091 source "$STATE_DIR/venv/bin/activate" pip install --quiet --disable-pip-version-check -r requirements.txt record "pip_freeze_sha $(pip freeze | sha256sum | cut -d' ' -f1)" 

verify.py

The reference probe separates transport errors from HTTP errors, because the two require different remediation.

import os, time, urllib.error, urllib.request BASE = os.environ["AGENT_BASE_URL"].rstrip("/") TOKEN = os.environ.get("AGENT_TOKEN", "") def classify(code: int) -> str: if code in (401, 403): return "auth" if code == 429: return "rate_limit" if 500 <= code < 600: return "cold_start" return "contract_mismatch" def probe(path: str = "/health", timeout: int = 5) -> dict: req = urllib.request.Request(BASE + path, headers={"Authorization": f"Bearer {TOKEN}"}) started = time.monotonic() try: with urllib.request.urlopen(req, timeout=timeout) as resp: return {"ok": True, "status": resp.status, "ms": int((time.monotonic() - started) * 1000)} except urllib.error.HTTPError as err: return {"ok": False, "status": err.code, "class": classify(err.code)} except (urllib.error.URLError, TimeoutError): return {"ok": False, "status": None, "class": "network"} if __name__ == "__main__": print(probe()) 

Failure triage table

The strongest submissions include a table like this, and each row cites the receipt timestamp where it occurred.

Where the free tier and free server fit

The task needs a target that a reviewer can afford to rebuild many times, which is why a free tier is useful here rather than decorative. MonkeyCode's free model access and free server option let a reviewer stand up a fresh instance per submission and throw it away afterwards. Because availability terms change over time, the reference target should be re-verified at the start of each hiring round rather than copied from an old post.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Treat the free tier as a measured environment, not a guaranteed one: candidates should record what they observed, when they observed it, and what they would do if that boundary moved. Candidates who claim unlimited or permanent access are failing the limits-honesty dimension, not the infrastructure one.

Common failure modes

1. README literalism. The candidate follows a stale instruction like a fixed Python version and never notes that the environment moved on. Good submissions record the contradiction and their resolution.
2. Silent fallbacks. A wrapper swallows 429 and retries forever, so the receipt shows success while the actual request never completed.
3. Missing receipts. Without an ordered command log, the reviewer cannot distinguish a lucky first try from a methodical rebuild.
4. Over-remediation. A single 502 triggers a full re-provision, which discards the very evidence the task is asking for.
5. Leaked resources. The submission leaves a background process or a running instance behind, and the teardown document omits both.
6. Source edits. The candidate patches the client to match their environment instead of adapting the environment to the client.

Reviewer checklist before scoring

• Re-run setup.sh from a clean home directory and confirm identical outcomes.
• Replay three receipt lines and check they are still valid commands.
• Open TRIAGE.md and verify each class maps to a different corrective action.
• Confirm no tokens, keys, or personal paths appear in the receipt.
• Follow TEARDOWN.md literally, then check for leftovers.

Limitations and who should not use this

This exercise measures environment reasoning, documentation discipline, and failure classification; it says nothing about model quality, algorithmic depth, or product taste. Teams hiring primarily for research or algorithm roles should pair it with a separate problem rather than stretch this one. Candidates with limited time or unstable home connectivity may be penalized for reasons unrelated to skill, so provide a stable sandbox and a shorter variant when needed.

The restore take-home is cheap to run and hard to fake, because the evidence is a replayable list of commands rather than a polished demo. A reviewer who cannot rebuild the environment from the submission alone should score that submission accordingly.