The best AI tools for forensic accountants test what the model read against the balances the bank printed, because a language model can misread one figure on a statement and still hand back perfectly valid JSON. That single test is what ZeraBooks' ranking of 9 AI tools for forensic work sorts on, and you can run it on any model's output in a few dozen lines.
We make ZeraBooks, and it sits first on that list. Its AI reads bank, card, check and brokerage statements, then proves every row against the printed balances before anyone downloads a file. The harness below does the same kind of check with no ZeraBooks code involved.
Why isn't schema validation enough?
Because schema validation tells you the shape is right, not the numbers. A model that reads 4,781.80 as 4,718.80 passes every JSON schema you can write. The bank already printed the answer key on the statement: an opening balance and a closing balance. Everything the model extracted has to connect those two.
What does the model output look like?
Here's a summary-level extraction for checking ••4417, March 2025, from Matter 25-117, a made-up matter. One figure is wrong on purpose.
import json from decimal import Decimal as D, InvalidOperation # What the model returned for checking 4417, March 2025. # One figure is wrong on purpose: money_out should be 4781.80. model_output = """ { "account_last4": "4417", "period": "2025-03", "opening_balance": "14902.35", "total_money_in": "8120.00", "total_money_out": "4718.80", "closing_balance": "18240.55", "checks": [ {"number": "3285", "date": "2025-03-14", "payee": "Harbor Marine Supply", "amount": "2450.00"} ] } """ Ask the model for amounts as strings. Floats in JSON get parsed as floats, and you lose the cents before you've checked anything.
How do you parse and tie it out?
REQUIRED = ["opening_balance", "total_money_in", "total_money_out", "closing_balance"] def parse(raw): data = json.loads(raw) missing = [k for k in REQUIRED if k not in data] if missing: raise ValueError(f"model left out {missing}") try: return data, {k: D(data[k]) for k in REQUIRED} except InvalidOperation as exc: raise ValueError("a balance is not a plain number") from exc def tie_out(v): computed = v["opening_balance"] + v["total_money_in"] - v["total_money_out"] return computed, v["closing_balance"] - computed Can the size of the gap say what went wrong?
Often, yes. A few patterns show up again and again in statement extraction:
def diagnose(diff, amounts=()): hints = [] cents = abs(diff) * 100 if diff == 0: return ["ties to the cent"] if cents % 9 == 0: hints.append("gap divides by 9: look for two swapped digits") for a in amounts: if abs(diff) == 2 * abs(a): hints.append(f"gap is twice {a}: that row's sign is flipped") elif abs(diff) == abs(a): hints.append(f"gap equals {a}: that row is missing or doubled") return hints data, values = parse(model_output) computed, diff = tie_out(values) print("computed closing:", computed) print("printed closing: ", values["closing_balance"]) print("difference: ", diff) for hint in diagnose(diff): print(" -", hint) Output:
computed closing: 18303.55 printed closing: 18240.55 difference: -63.00 - gap divides by 9: look for two swapped digits The 8 and the 1 swapped places, and the gap is 63.00, which is 7 × 9. Any swap of two adjacent digits leaves a gap that divides by 9, so this hint is cheap and right more often than you'd expect. It isn't proof. Plenty of other gaps divide by 9 too.
Now flip the sign on check 3285, the $2,450.00 payment to Harbor Marine Supply:
print(diagnose(D("4900.00"), [D("2450.00")])) # ["gap is twice 2450.00: that row's sign is flipped"] A model that files a check as money in moves the balance by double its amount. Row-level extraction gives you the amounts list to test against.
What should a harness like this include?
- A tie-out per account and, on card statements, per cardholder, since a card statement can tie overall while one cardholder's rows are off.
- A balance chain across months, so a statement the model never saw shows up as a break.
- The same statement run more than once. If two runs disagree, at least one of them is wrong, and you know before the tie-out tells you.
- A stop rule. A statement that doesn't tie goes to a person, not into the workpaper.
That last one is the whole point. In ZeraBooks, a statement that misses by a cent is marked Needs a look instead of moving on.
Where does this leave general chatbots?
A general chatbot can read a statement you paste into it. It doesn't prove the result against the printed balances, and client confidentiality is worth thinking about before you upload a production anywhere. If you want the full picture of the checks a matter needs beyond extraction, here's how ZeraBooks handles a forensic matter, check images and duplicate uploads included.
Let the model read. Let the bank's own numbers grade it.