The earlier post listed five tests for an agent-ready CRM: revert in one action, pause one agent, an audit log that names who acted, a reply that stops the follow-up, and mail you can see before it leaves. Three readers came back with cases it missed. Each one is a better test than the five in that list, so here they are.

Test 6: take the API key away (from @autenai)

Not every agent uses the API. Computer-use agents drive the web UI in a signed-in browser, so the key-based controls in tests 2 and 3 never come into play.

Run it: sign in with a throwaway account, hand the browser to the agent, have it edit five records and delete one.

Pass: the log says an agent did it, separate from you. You can cut that agent off mid-run without signing yourself out. The delete can be undone.

Watch for: every action logged under the person whose session it is. Then pause, revoke and attribution all quietly assume the agent has a key. If the agent has to share your login, the honest scorecard answer is "not covered". The workaround is a second user (or browser profile) that exists only for the agent, so it can be told apart and switched off on its own.

Test 7: hit the rate limit halfway through a bulk update (from @junfengzhou01)

Give the agent a key with a low limit and have it update 200 records in a loop.

Pass: the call that hits the limit is refused with a status and a retry hint the agent can read, not a 500 or an HTML page. Nothing is half-applied inside a single record. Afterwards you can list exactly which records changed, by run, and revert only those. Retrying the whole loop with the same idempotency keys does not duplicate the ones that already went through.

Watch for: a bulk endpoint that returns 200 while quietly skipping the tail, a limit that answers in a format the agent cannot parse and so retries in a tight loop, and a log with no run id, where "which ones changed" means diffing a spreadsheet.

Test 8: stop one agent of several (from @Considus on Mastodon)

Considus built a stop file that halts an agent mid-run, and asked how a single server tells which agent to stop. Running one server instance per agent makes the switch easy, because the file belongs to one process. It also gives you N servers to keep in step.

Run it: two agents, one server. Start both on a long job and stop one.

Pass: the stopped agent's next call fails with a message that says why, and the other keeps going. Nothing restarts. The identity the server uses to decide is the credential on the call, not the process.

Watch for: a stop that is a flag the assistant can read and set itself, and a stop that only takes effect when the agent next asks for confirmation.

Scorecard additions

  • An agent in the browser is told apart from the person, and can be cut off alone.
  • A rate-limited agent gets a readable refusal, and you can list what changed.
  • One agent of several can be stopped on its next call.

Same rule as before: run them on a fresh workspace with throwaway data. A ninth test that caught something these did not is worth a comment.

From Deskhand, by Grain64.