Why the model shouldn't be the one that says "done"

Disclosure: this article was written with AI assistance and reviewed and edited by me, the tool's author. Loopsmith is my own open-source project.

I have a few jobs I redo every week and care about getting right: a competitor roundup, a lead list, a research brief. Handing them to an AI agent works up to a point. The part that kept failing for me was the end of the job, not the work itself.

The agent finishes, says "done", and I open the output to find a missing section, a citation that points nowhere, or a test it quietly edited so it would pass.

So I built a small tool around one rule. The model never gets to say "done". I call it loopsmith: a skill that creates self-evolving, loop-based, purpose-specific skills, backed by a Rust CLI. It's MIT-licensed, at v1.0.0, and I'm sharing it here to find out whether the idea holds up for other people.

20-second demo: watch it with sound.

The problem with self-certified completion

Most agent loops end in one of two ways. The model decides it's finished, or a counter runs out. The first means the thing doing the work also grades the work. The second means you get whatever it had when the budget ended.

A model asking itself "is this done?" is not a check. It's an opinion from the party with the most reason to be agreeable. The design rule that follows is simple: whatever certifies completion must not be the thing doing the work.

What loopsmith does

You describe a job once, in one plain text file (YAML or Markdown). The file says what you want, what proves it, and every way the run is allowed to end. loopsmith then:

puts an AI to work on it,
runs a checker over the actual output files,
sends the work back if the checker says no,
stops when the checker says yes, or when a limit is hit.

That checker is the gate, and it's compiled Rust code. It reads files, not the model's description of them. In the README's words, goal_satisfied is written by the deterministic gate and by nothing else.

The gate can also revoke. If you delete a required artifact later, a goal that was satisfied flips back to unsatisfied. A system that can only mark things complete is a burndown chart with extra steps, and I didn't want that.

A judge running on the same provider as the work it grades is refused, not just warned about. A model shouldn't mark its own homework.

"Done" is only half of stopping

An unattended loop needs more than a good definition of done. It needs to be bounded. loopsmith has ceilings on:

iterations,
cost,
tokens,
wall-clock time,

plus a no-progress cap for the loop that is still busy but no longer changing anything. Every iteration is checkpointed and written to an append-only ledger, so a run that dies at 4am resumes instead of restarting. You can ask the ledger what happened afterwards:

loopsmith run ledger <config> <run-id>

A skill that makes skills

The other half of the project is a skill, installed into a coding agent, that creates self-evolving, loop-based, purpose-specific skills. You describe a purpose, and it helps you shape a loop around it: one skill per job, with the gate that proves the job is done.

npx skills add bitphill/loopsmith

The skills drive the loopsmith binary, so install the binary too.

What "self-evolving" means here

It means the loop improves from work that has to be redone. When the gate fails a goal, the work goes back to the AI with what failed, and the loop learns from that redo. It also trials candidate skills and scores them against gate outcomes, and it keeps scratchpad notes between iterations. A candidate with too few trials is recorded and ignored, so one lucky run isn't treated as evidence.

What it does not do is move its own goalposts. It never changes its own goals, checks or config without your approval. Changes to goals, validations, success criteria and the skills a config uses arrive as proposals, and you apply them yourself:

loopsmith run proposals <config> <run-id>

A system that can move its own goalposts can't certify that it met them, so I didn't build one.

Install

Pick whichever package manager you already have. All of them put a loopsmith on your PATH.

# crates.io, builds from source cargo install loopsmith # npm, downloads a prebuilt binary npm install -g @bitphill/loopsmith # PyPI, downloads a prebuilt binary on first run pip install loopsmith-cli # Homebrew brew install bitphill/loopsmith/loopsmith

On Homebrew 6, third-party taps need a one-time brew trust bitphill/loopsmith first. There is also ./install.sh (Linux, macOS, BSD) and install.bat (Windows) in the repo, and the npx skills add line above for the agent skills.

The npm and PyPI names differ from the command because plain loopsmith was already taken on both by unrelated projects.

Three ways to build a loop

You don't have to hand-write the config. All three front ends build the same file:

loopsmith --guided # question-and-answer wizard in the terminal, works over SSH loopsmith --web # local web UI, bound to 127.0.0.1 only loopsmith loop new -p ~/loops/my-loop

There's also a local MCP server, so an agent can drive it directly.

Fifteen working example loops ship with the binary. Loading one is the fastest way to see what a validation or a stop gate looks like. A few of them:

Research brief: refused until every claim has a citation a script can check.
Refactor: two builders on one module, the test suite as the gate, and edits to the tests forbidden.
Landing page: held until it is accessible, fast, and actually deploys.
Lead list: stops when the list is long enough and clean enough.

What it isn't

I'd rather be plain about this:

It doesn't make a bad model good. It makes a bad result visible, and it stops you paying for the same mistake forever.
A gate is only as good as the checks you write. If your definition of "good" is vague, the gate will be vague too. Writing it down is the real work.
It's v1.0.0 and has had very few users besides me. I expect rough edges.