In comparetiers, I built a SaaS price-comparison agent that makes eight model calls and runs three tools.
But what if the process dies halfway? Everything is lost!
So I changed its state to be an event journal, so:
- you can stop this agent at any moment and run
resume - you can replay a whole run without calling the model
- the journal is an audit trail, line by line
The code is written in TypeScript and I use qwen3:8b with Ollama.
Initial problem: state in local variables
My first version of the agent loop looked like this:
let turn = 1; const MAX_TURNS = 8; const messages = [system, user]; while (turn <= MAX_TURNS) { const decision = await callModel(messages); const result = await runTool(decision.toolCall); messages.push(decision, result); turn++; } All the state (messages, turn counter, amounts seen so far) lives on the stack. There's no resume point and no reliable history.
The model: one event per step
I represented a run as a JSONL file. Each line is an event, validated by zod.
run_started -> turn_started -> model_decision -> tool_called -> tool_result -> ... -> run_ended Plus model_failed, tool_error, and a run_resumed on each resume. A real excerpt:
{"seq":1,"kind":"run_started","question":"How much is Notion Plus for 5 seats, billed monthly?","maxTurns":8,"model":"qwen3:8b"} {"seq":2,"kind":"turn_started","turn":1} {"seq":3,"kind":"model_decision","turn":1,"toolCall":{"name":"get_plans","arguments":{"product":"notion"}},"usage":{"promptTokens":514,"outputTokens":22,"durationMs":9272}} The schema is a discriminated union on kind. Every line read back is parsed again, so a corrupt journal fails loudly instead of producing a strange state.
Important: turn_started is written BEFORE the model call so a turn cut off mid-call shows up in the journal and counts against the budget. Crashing doesn't buy extra turns.
The durability rule: write, fsync, then act
So an event is written and synced to a storage before the agent acts on it:
import { closeSync, fsyncSync, openSync, writeSync } from "node:fs"; export const appendEvent = (path: string) => (event: Event): void => { const fd = openSync(path, "a"); try { writeSync(fd, `${JSON.stringify(event)}\n`); fsyncSync(fd); } finally { closeSync(fd); } }; const runId = "r-1"; const append = appendEvent("./runs/${runId}.jsonl"); append({ seq: 2, kind: "turn_started", turn: 1 }); append({ seq: 4, kind: "tool_called", turn: 1, toolCall: { name: "get_plans", arguments: { product: "notion" } } }); tool_called is on disk before the tool runs, tool_result is on disk before the next model call, so on resume there are only a handful of cases, depending on the last event:
| last event | resume does |
|---|---|
tool_called | runs the tool again (all tools are read-only) |
model_decision with a tool | writes tool_called, runs the tool |
turn_started, model_failed | new turn; the lost one is already counted |
a result, or run_started | applies the stop rules, then a new turn or run_ended |
run_ended | refuses (exit 3) |
There's no other case to handle.
A precision for the first row: running a tool again is only safe because all tools are read-only. The guarantee is "at least once", not "exactly once". A tool with a side effect (a payment, sending an email) would need an idempotency key so it doesn't run twice.
State is fold(events)
The agent's current state is a pure fonction of the journal:
export const fold = (runId: string, events: readonly Event[]): AgentState => ... It's deterministic: same events = same state. Start, resume, the listcommand, replay,... all go through it. There's no second copy of the state that could drift. That's the core idea of event sourcing, and it carries over to an agent unchanged.
Two edge cases need handling:
- Torn line: a crash during a write leaves an incomplete last line. It's dropped with a warning, and the file is rewritten without it (temp file, then
rename) before the next append. An invalid line anywhere else is a hard error that names theseq. - Lock:
<runId>.lockholds the pid driving the run. If that pid is alive,resumerefuses (exit 4). If it's dead, which is what aSIGKILLleaves behind, the lock is taken over and the old pid is recorded.
Replay: the run without the model
Replay runs the same loop, but with a decide function that returns the decisions recorded in the journal instead of calling the LLM. By default, tool results also come from the journal. With a flag --live-tools, the tools actually run against the current catalog, and a divergence means the data has changed since the run. Every event produced is compared to the journal (kind and content, ignoring timestamps and token usage), and replay stops at the first divergence.
That catches anything the code drifts on: tool_called arguments that no longer match the decision, a changed stop rule, a final result that no longer leads to the recorded run_ended. It's a regression test built from real runs.
What it doesn't catch with recorded results: a tool_result edited by hand, since it's replayed as-is.
What it costs, what it buys
The journal only grows: on long runs, you'd want snapshots so a resume doesn't fold everything from scratch. Changing an event's schema breaks reading older journals, so events will need versioning. And resume only holds for tools that are safe to run twice.
In return, a crash costs nothing anymore: resume picks up from the last event without paying again for the turns already done. And every recorded run becomes a free regression test, replayed without a single model call.
