The Forced Swap: A Bootcamp Lab Where the App Has to Survive a Model Change Mid-Run

A team that can only run against one model doesn't have an agent. It has a demo with a vendor-shaped hole in the middle.

So this lab grades the seam, not the demo. Students build a thin adapter, probe what the endpoint can actually do, then swap the model underneath a running pipeline and prove the app either survives or fails loudly.

It takes 30–45 minutes, needs two callable free model endpoints and one deployed instance, and produces artifacts an instructor can inspect without trusting anybody's summary.

Setup

Give each team:

1. Two model endpoints with keys, reachable from a laptop.
2. One deployed instance they control (the free server option is enough).
3. A fixture set of roughly ten small tasks with expected JSON shapes.
4. Node 20+ and nothing else.

This lab was written to be runnable with MonkeyCode's free model access and its free server option, both offered at no cost as part of the project's outreach. Confirm the current allowance and terms in the project's own docs before you build a semester plan on them — free-tier limits are terms, not constants. The preset describes an allowance on the order of ten million tokens; treat that as a number to verify, not a value to hard-code into a rubric.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Checkpoint 1 — Probe the endpoint instead of assuming

Most agent breakage in student projects is an assumption, not a bug: someone assumed JSON mode, or tool calls, or a context size, and never checked. The first checkpoint makes the assumption visible and timestamped.

// scripts/probe.mjs — illustrative, not executed here. // Adjust the path and headers to whatever your provider's docs specify. import { writeFile } from "node:fs/promises"; const BASE = process.env.MODEL_BASE_URL; const KEY = process.env.MODEL_API_KEY; const MODEL = process.env.MODEL_ID; if (!BASE || !KEY || !MODEL) { console.error("missing MODEL_BASE_URL / MODEL_API_KEY / MODEL_ID"); process.exit(2); } async function chat(messages, extra = {}) { const res = await fetch(BASE, { method: "POST", headers: { "content-type": "application/json", authorization: `Bearer ${KEY}`, }, body: JSON.stringify({ model: MODEL, messages, ...extra }), }); if (!res.ok) throw new Error(`${res.status} ${await res.text()}`); return res.json(); } const textOf = (p) => p?.choices?.[0]?.message?.content ?? ""; const results = { model: MODEL, at: new Date().toISOString() }; // 1. Plain completion. try { await chat([{ role: "user", content: "Reply with the single word: ok" }]); results.plain = "pass"; } catch (e) { results.plain = `fail: ${e.message.slice(0, 120)}`; } // 2. Structured output: three attempts, because one lucky parse is not a capability. results.jsonOnly = []; for (let i = 0; i < 3; i++) { try { const out = textOf( await chat([ { role: "system", content: "Respond with JSON only. No prose, no fences." }, { role: "user", content: 'Return {"files":[],"summary":"empty"}' }, ]) ); JSON.parse(out); results.jsonOnly.push("pass"); } catch { results.jsonOnly.push("fail"); } } // 3. Tool-call shape: presence is a capability, absence is a design input. try { const payload = await chat([{ role: "user", content: "Weather in Lisbon?" }], { tools: [ { type: "function", function: { name: "get_weather", parameters: { type: "object", properties: { city: { type: "string" } }, required: ["city"], }, }, }, ], }); results.toolCalls = payload?.choices?.[0]?.message?.tool_calls?.length ? "pass" : "absent"; } catch (e) { results.toolCalls = `absent: ${e.message.slice(0, 80)}`; } await writeFile("probes.json", JSON.stringify(results, null, 2)); console.log(results); 

Run it once per endpoint and commit the two probes.json files. The rule for the rubric: a capability claim without a probe file earns zero, even if it happens to be true.

Checkpoint 2 — Put a seam at the model boundary

The adapter below is deliberately boring. It reads capabilities from the probe output and converts "this endpoint can't do JSON" into a retry strategy rather than a crash.

// src/model.js — illustrative. export class ModelShapeError extends Error { constructor(id, cause) { super(`model ${id} did not produce parseable output`); this.name = "ModelShapeError"; this.cause = cause; } } const stripFences = (s) => s.trim().replace(/^``` {% endraw %} (?:json)?/i, "").replace(/ {% raw %} ```$/, "").trim(); const JSON_RETRIES = 2; export function createModel({ id, jsonMode, callModel }) { return { id, async complete(prompt, { wantJson = false } = {}) { const needsScaffold = wantJson && !jsonMode; const messages = needsScaffold ? [ { role: "system", content: "Respond with JSON only, no fences." }, { role: "user", content: prompt }, ] : [{ role: "user", content: prompt }]; let lastError; for (let attempt = 0; attempt <= (wantJson ? JSON_RETRIES : 0); attempt++) { const text = await callModel(id, messages); if (!wantJson) return text; try { return JSON.parse(stripFences(text)); } catch (e) { lastError = e; } } throw new ModelShapeError(id, lastError); }, }; } 

Two grading details matter here. First, jsonMode must be wired from probes.json, not from a comment. Second, ModelShapeError must reach the caller — a try/catch that returns {} is an automatic fail, because it turns a loud failure into a silent one.

Checkpoint 3 — Swap mid-run and diff the runs

The swap is where the lab earns its name. Students run the same fixtures twice, changing only MODEL_ID, then explain every difference.

MODEL_ID=$FREE_MODEL_A node harness.mjs --fixtures fixtures/ --out runs/a.json MODEL_ID=$FREE_MODEL_B node harness.mjs --fixtures fixtures/ --out runs/b.json node scripts/diff-runs.mjs runs/a.json runs/b.json 

A run passes when, for every fixture, it produces the same JSON shape and the same pass/fail verdict, or it raises a typed error naming the model. It fails when outputs drift silently: empty arrays where data should be, null fields that downstream code treats as valid, or a "success" missing required keys.

The diff script can be twenty lines. Compare keys, not prose.

Checkpoint 4 — Measure the free server honestly

Free servers are allowed to be slow, and often are, especially on a first request after an idle period. Students should record what they observe instead of quoting a blog post.

# Sample once after a known idle window, then again right after a warm request. for i in 1 2 3; do curl -sS -o /dev/null \ -w "%{http_code} starttransfer=%{time_starttransfer}s total=%{time_total}s\n" \ "$APP_URL/healthz" sleep 1 done 

Design requirement: /healthz must separate "process is up" from "dependencies are ready". A 200 that hides an unready database is worse than a 503. Pair this with a skeleton state in the UI so a slow first request looks like loading rather than breakage.

Do not publish numbers you did not record. Idle behavior on any free tier is a term that can change, not a physical constant.

Grading rubric

Two rubric rules keep it fair. No points for guessing correctly. And a team that documents a limitation it could not fix scores higher than a team that hides it behind a catch block.

Stretch goals

• Add a third endpoint and let the harness pick one per fixture, with the choice logged.
• Cache probe results with a short TTL and re-probe on change, so capability data does not rot.
• Add a circuit breaker that opens after repeated ModelShapeErrors and falls back to a plain-text path.
• Log approximate token spend per fixture so a team can see which task is expensive.

Where this lab breaks

Two free endpoints may share a failure mode, so the lab proves seam durability, not provider diversity. Probe results are point-in-time and go stale. Ten fixtures are a smoke test, not a benchmark — never quote the pass rate as a model quality score. And cold-start samples reflect one moment on one tier.

Who should skip it

Skip this if you need an availability commitment, if students cannot obtain two keys, or if your course is really about prompt wording rather than system boundaries. This lab will not teach prompt craft, and it should not be used to plan production capacity.

How to check the work is real

1. probes.json exists for both endpoints and the timestamp is recent.
2. runs/a.json and runs/b.json exist, and the diff output is committed.
3. The cold-start log contains at least one idle sample and one warm sample.
4. A fresh clone reproduces the run with the documented command.
5. grep -r "fetch(" src/ returns only the adapter file.

If you want to run this lab without provisioning anything, start with MonkeyCode's free model access and the free server option, then check the current terms in the project docs before you schedule it. The seam you build is yours either way.