This month OpenAI announced Dots, personal agents that keep working when you're no longer in ChatGPT. SpaceXAI is building a marketplace of specialised Grok bots. "The agent runs in the background" is quickly becoming the default.

As soon as an agent runs without you watching, it's no longer a chat. It's a background worker. And we've known for a long time how background workers fail.

At-least-once means "sometimes twice"

I wrote about this for Kafka consumers in September. Any system that retries will, sooner or later, do the same work twice: the worker crashes after doing the work but before recording that it did, and the retry runs it again.

For a Kafka consumer, "twice" might mean a duplicate row. For an agent, it means:

  • the same email sent twice
  • the same refund issued twice
  • the same calendar invite, PR, or payment, twice

An agent loop is a retry loop with a language model inside it. The model times out, the tool call returns a 502, the session restarts after a deploy. Each one is a chance to repeat an action that already happened.

The fix is the same one

Give every side effect an idempotency key, and make the receiving side remember it.

// the key comes from the *intent*, not the attempt const key = hash(taskId, 'send_invoice', invoiceId); await tools.sendEmail({ to, subject, body, idempotencyKey: key });

If the agent restarts and tries again, the key is the same, so the email service (or your wrapper around it) returns the first result instead of sending again.

Two rules make this work:

  1. The key comes from what you're trying to do, never from the attempt. A random UUID per call defeats the purpose.
  2. The check and the write happen together. "Have I done this?" and "record that I did it" in one transaction, or a unique constraint that makes the second insert fail. Postgres is better at keeping promises than application code is.

Poison tasks need a dead-letter queue

The other Kafka lesson: some messages will never succeed. Maybe a bad input, or a tool that always rejects it. A consumer that retries forever blocks everything behind it.

An agent that retries forever does something worse: it keeps trying new approaches. Each one costs money, and some of them are creative. Agents need what consumers have:

  • a retry limit per task
  • a dead-letter state a human looks at
  • a replay that re-runs from a known point, with the same idempotency keys, so it can't double-act

Write down what the agent did, in a log it can't edit

In my parking app the money ledger is append-only: a correction is a new reversing entry, never an edit. An agent acting for a person should leave the same kind of trail. Every tool call, its arguments, its result, its key. When someone asks "why did it send that?", you can answer from the record instead of guessing.

What I'd tell you before you start

"Agent" is a new word for an old shape. If it runs when nobody's watching, it's a worker. Give it idempotency keys, a retry limit, a dead-letter queue and an audit log, the same as any consumer you'd trust with money.

Related: my earlier posts "At-Least-Once Means Sometimes Twice" and "Poison Messages in Kafka".