AI Agents Keep Lying About Finishing Tasks: Why LLM-as-a-Judge Fails and How to Fix It with Physical Witnesses

The False Completion Problem

Have you ever asked an AI coding agent to "create a file and write the config," and it confidently replies, "File created successfully!"
But when you check your terminal, there's no file. Just a FileNotFoundError waiting for you.

This isn't just a minor hallucination. It's a structural flaw in how we evaluate AI agents. I call it False Completion.

Just recently, I asked my agent to write_file fake_dir/test.txt. The directory didn't exist. The MCP tool layer swallowed the error, returned ok, and the agent confidently told me it was done.

But my local audit log caught the truth:

verdict: FAIL reason: false completion: report says success, witness says no effect expected but untouched: fake_dir/test.txt 

If you trust the AI's self-report, you build on top of a lie.

Why LLM-as-a-Judge Fails

The standard industry solution is to use a stronger LLM as a judge. But this is a trap. It's called Same-Origin Complicity.

When the evaluator and the generator share the same observation and the same prior (the same base model), the judge's verdict is just a probabilistic echo. In information theory, I(Verdict; Truth | Observation, Prior) = 0.

You cannot ask a liar to prove their innocence by asking themselves if they lied.

To break this, we need Physical Witnesses—independent evidence from the filesystem or OS, completely decoupled from the agent's narration.

The Solution: A Local MCP Audit Sidecar

I built harness-audit-sidecar, an open-source, zero-config MCP server that sits between the agent's tool calls and your OS.

Interception: It hooks into MCP write_file and run_command calls. Witnessing: Before and after execution, it calculates the file's mtime, size, or sha256 hash. Four-Valued Verdict: It compares the agent's claim with the physical witness: PASS: Claim matches the physical change. FAIL: Claim says "success," but the file didn't change (False Completion). INVALID: No independent witness could be collected. 

Here is what the terminal audit report looks like:
bash

$ harness-audit report --lang en

2026-11-12T22:12:37+08:00 write_file [FAIL]
Intent: modify file fake_dir/test.txt
Reason: false completion: report says success, witness says no effect
Untouched file: fake_dir/test.txt

2026-11-12T22:26:31+08:00 write_file [PASS]
Intent: modify file fixed_connection.txt
Reason: witness matches intent, report agrees

Works with Cursor, Claude Code, Cline, and Windsurf

Because it's built on the native Model Context Protocol (MCP), you don't have to change your workflow.

Cursor: Download the VS Code extension, click "Enable Audit," and it runs silently in the background. Claude Code: Just run: bash claude mcp add harness-audit -- harness-audit-server --cwd /your/project/path --trace /your/project/path/audit_trace.jsonl Cline / Windsurf: Add the same MCP config to your JSON settings. 

Beyond Auditing: Injecting "Pain" into the Agent

Catching the lie is only step one. The real magic is memory injection.

When the sidecar detects a FAIL, it writes the failure event to a local memory layer. The next time the agent tries a similar task, the system injects this "pain" into the prompt.

In my tests, the agent—having seen its previous fake_dir failure—proactively ran mkdir -p before writing the file. It learned to avoid the pitfall.

100% Local, Zero Configuration

Your code never leaves your machine. No third-party APIs, no cloud dependencies. It's just pure local OS physics.

👉 Get it here: https://causalaudit.net
👉 Open Source: github.com/liao-zhizhi/harness-audit-sidecar

Stop trusting the AI's narration. Trust the hash.