Picture a release with a perfect paper trail. You have the reviewed commit, the resolved dependencies, the builder identity, the artifact digest, and a green record of every check that ran. All of it is signed.

It can still contain a bug that lets anyone mark an invoice as paid.

Provenance is worth having, and AI-assisted development makes it more useful, because teams increasingly need to reconstruct how a change came together. The trouble starts when a complete record begins to feel like evidence that the code is good.

The audit trail vs. the actual behavior

Provenance answers questions like these:

  • Which commit is in this artifact?
  • Who approved it?
  • Which dependency versions were resolved?
  • Which build produced it, and which checks ran?

Those answers are valuable during an incident, and they help you reproduce a release. None of them tells you whether the code does the right thing.

The scenarios below are illustrative, not real incidents.

Failure mode 1: faithfully recording the wrong code

An assistant helps write a payment webhook:

app.post("/webhooks/payments", (req, res) => { const sig = req.get("X-Signature"); if (sig && !verifySignature(req.rawBody, sig)) { return res.sendStatus(401); } markInvoicePaid(req.body.invoiceId); res.sendStatus(200); });

A bad signature gets rejected. A request with no signature at all skips the check and marks the invoice paid.

The build can be hermetic and the attestation can verify cleanly. The record is accurate: this exact code, built this exact way, reached production. Build integrity tells you the artifact wasn't tampered with on the way there, and frameworks like SLSA are aimed at that problem. Whether the source was right to begin with is a separate question.

Failure mode 2: "checks ran" gets read as "checks were enough"

The tests for that webhook might cover a valid signature and an invalid one. Both pass, so the record says unit-tests: pass, and that's true.

The missing case is the one that matters:

test("rejects a webhook with no signature", async () => { const res = await request(app) .post("/webhooks/payments") .send({ invoiceId: "inv_123" }); expect(res.status).toBe(401); });

A provenance record can show that a job ran and passed. It can't show what the job never tried.

Failure mode 3: a record of review isn't a record of scrutiny

approved_by: reviewer-a is a fact about a click. It doesn't tell you whether the reviewer traced the unsigned path, or spent four minutes on a large AI-assisted diff near the end of the day.

Keep the approval record. Just don't let it stand in for the review itself. On sensitive changes, the reviewer should be able to answer a specific question: what happens when this header is missing?

Failure mode 4: a clean scan is a snapshot

A dependency scan can pass on release day, and a vulnerability in one of those packages can be published a week later. The record of that clean scan stays true while the risk changes.

Here, provenance earns its keep. If you recorded exactly what shipped, you can answer "are we affected?" in minutes instead of guessing. Just treat the old scan result as a dated observation, and keep scanning what's already deployed.

Failure mode 5: the audit log becomes the leak

It's tempting to capture everything an AI tool touched, including full prompts. But prompts pick up whatever a developer pasted in: a stack trace with a customer's email address, a production log, a connection string.

Now that material sits in a long-lived record that may be widely readable, kept in the name of traceability.

Most incident questions don't need the raw prompt. Something like this usually goes a long way:

{ "artifact_digest": "sha256:...", "commit": "9f3c1ab", "pull_request": 4821, "approved_by": ["reviewer-a"], "lockfile_sha256": "...", "builder": "ci/release.yml@refs/tags/v2.14.0", "checks": [ { "name": "unit-tests", "result": "pass" }, { "name": "static-analysis", "tool_version": "...", "result": "pass" } ], "ai_assisted": true, "ai_tool": "assistant-name@version" }

That shape is only an example. If you want a standard format, look at SLSA provenance and in-toto attestations. If your team does need to keep prompts, redact them, limit who can read them, and give them a retention period.

What goes in the record, and what still has to happen

Record what helps you explain and reproduce a release:

  • Commit, PR, and approvers
  • Resolved dependencies
  • Builder identity and artifact digest
  • Which checks ran, with tool versions and results
  • Whether AI tools were involved, and which ones

Then do the work the record can't do for you:

  • Code review: is this the right behavior?
  • Testing: does it do what the requirement says, including what it has to refuse?
  • Security analysis: are there risky patterns, data flows, vulnerable dependencies, or committed secrets?

Provenance is the part that lets you trace and reproduce a release. Those three are how you find out whether it should have shipped.

A signed record tells you where the software came from. It takes a different kind of work to know whether it's safe.

What does your team actually record for AI-assisted changes, and has any of it helped you during a real incident? Do you keep prompts, or did you decide against it?