A trace contains six records with the same tool name. How many times did the agent try the operation?
Six is a tempting answer. It might also be three calls with start and finish records, two calls with duplicated telemetry, or six unrelated operations that happen to use the same tool.
Counting rows is easy. Counting attempts requires an identity model.
I maintain AgentInspect. The examples here were tested against 6.31.35 using a synthetic read operation. This is about interpreting execution evidence, not implementing live retries or promising exactly-once effects.
Give each identifier one job
The following vocabulary works even if your tracing library uses different field names:
| Identity | Meaning | Changes on a retry? |
|---|---|---|
| Run | One captured execution | Usually not within that execution |
| Operation | One logical request being attempted | No |
| Attempt | One execution attempt of that operation | Yes |
| Event or delivery | One observation or its transport occurrence | Potentially |
| Idempotency key | A downstream service's effect-deduplication scope | Defined by that service's contract |
An attempt can emit multiple events. One event can be delivered more than once. One operation can span several attempts. These are relationships, not interchangeable counters.
Transport duplication and repeated application execution must remain distinguishable.
Record identity where the operation is invoked
Install the pinned release:
npm install --save-exact agent-inspect@6.31.35 Use the metadata-bearing step() API. In this version, the step.tool() shortcut accepts the name and function, not a third metadata argument.
import { inspectRun, step } from "agent-inspect"; function attempt(attemptId, attemptNumber, fn, retryOf) { return step("lookup_policy", fn, { type: "tool", metadata: { toolName: "lookup_policy", operationId: "fixture-op-1", attemptId, attemptNumber, ...(retryOf ? { retryOf } : {}), }, }); } await inspectRun("two-read-attempts", async () => { await attempt("a1", 1, async () => { throw new Error("Synthetic temporary failure"); }).catch(() => {}); return attempt("a2", 2, async () => ({ found: true }), "a1"); }, { traceDir: ".agent-inspect", silent: true }); The swallowed first error is deliberate fixture behavior. It is not a general error-handling recommendation. The fixture represents a bounded recovery path whose first attempt fails and second succeeds.
The IDs are static because the test runs in an isolated fixture. Real application identities need suitable uniqueness and scope. Don’t reuse fixture-op-1 across unrelated operations and expect the evaluator to infer the difference.
Ask the rule the question you mean
After opening the generated trace with openTraceFile, evaluate two contracts:
import { defineTraceContract, evaluateTraceContractRead, } from "agent-inspect/checks"; const oneAttempt = defineTraceContract({ retry: { maxAttempts: 1 } }); const twoAttempts = defineTraceContract({ retry: { maxAttempts: 2 } }); const strict = evaluateTraceContractRead(trace, oneAttempt); const bounded = evaluateTraceContractRead(trace, twoAttempts); For the source fixture, the first fails with contract.retry.max-attempts; the second passes. The result refers to attempts associated with the recorded logical operation, not the total number of JSONL rows.
The current implementation groups explicit retry identity within a run. It uses distinct attempt IDs when available and supports other documented evidence forms, including validated attempt numbering. Missing or conflicting identity is not permission to invent a retry relationship from a repeated name.
The important engineering decision comes before the rule: where does your application define the logical operation? An adapter cannot reliably reconstruct a business boundary you never recorded.
Redelivery is a useful negative control
The companion test copies the second attempt's exact terminal record and appends it to the source trace. It then evaluates the same contracts again.
| Source variation | Maximum one attempt | Maximum two attempts |
|---|---|---|
| Two recorded attempts | Fail | Pass |
| Same trace plus exact terminal redelivery | Fail | Pass |
This is a deliberately narrow claim. An identical redelivered record should not manufacture an extra attempt. It does not follow that every repeated-looking record can be discarded.
If the same declared identity appears with different arguments, status, or ownership, you have a conflict to investigate. Blindly applying new Map(events.map(e => [e.id, e])) can erase contradictory evidence by keeping whichever row arrived last.
A useful test set therefore includes both exact duplication and conflicting reuse. Keep their expected outcomes separate.
Don’t promote attempt counts into effect counts
Two attempts do not prove two writes. One timed-out attempt does not prove zero writes. A successful response does not necessarily prove the effect was observed by a recipient.
For a read operation, that distinction may only affect latency analysis. For payments, notifications, or reservations, it changes the recovery protocol.
An idempotencyKey recorded in metadata is evidence that a key was recorded. To claim that the destination deduplicated the effect, you need the destination's behavior and appropriately scoped receipts. AgentInspect's trace analysis is not that destination.
This is why I would keep retry accounting, effect verification, and authorization as separate checks. They may use related identifiers, but they answer different questions.
Check the identity after conversion, too
If you export traces, rerun the attempt-count check after reading the exported representation. A structurally valid span tree can lose operation metadata while retaining every tool name.
The included OTLP experiment preserves the retry verdict for this fixture when run status is evaluated separately. It also exposes a run-status discrepancy after recovery, documented in the companion interoperability article. Passing one invariant is not proof of complete semantic equivalence.
Start with one operation, two attempts, and one duplicate delivery. That tiny fixture is often more informative than a large trace full of plausible-looking counters.
Implementation references
- Retry-safety rule implementation
- The actual
step()and shortcut signatures - Companion:
examples/probes.test.mjs, testD02/H01.

