Claude for business decisions often come down to which model to put behind an agent, and this post shows how to settle that with a small test you can rebuild on your own data: an invented CRM with known traps, ground truth computed two ways, an agent loop with one read-only SQL tool, a scorer with a tolerance, and a cost per right answer instead of a cost per token. By the end you will be able to run the same comparison on your own questions and see what a right answer costs from each model. The trigger was the launch of [Claude Fable 5.2](https://www.anthropic.com/claude-fable-and-mythos-5-2) on 9 November, at $8 / $40 per million input and output tokens, twice [the list price](https://platform.claude.com/docs/en/about-claude/pricing) of Opus 5.5. I had Claude Code build the test, and a GitHub workflow made the API calls. Everything below is from the real files, and every code block is trimmed from them. ![Dark cover dated 9 November 2026, kicker Claude News: Claude Fable 5.2 is here. Sonnet added up the pipeline right every time](https://brinvik.com/claude-news/2026-11-09-claude-fable-pipeline/brinvik-pipeline-cover-en-1920x1080.png) ## The setup in one paragraph Three models (Claude Fable 5.2, Claude Opus 5.5, Claude Sonnet 5.5), the same 8 questions from an invented sales director, effort medium\, 2 runs per question per condition, and two conditions: with the export's column documentation and without it. That is 32 runs per model and 96 in all. Every call, every query and every answer is written to a JSONL file. The whole thing cost $9.58 at list price. ## Building a CRM with traps on purpose pack.py\ generates an invented Danish software firm, Saltholm Data ApS, deterministically, so the same seed always gives the same database. It has 150 live deals plus 12 archived copies left over from merged duplicates, amounts in 4 currencies, 9 deals with two CFOs in the contacts table, and 16 live deals that entered proposal more than once. Each of those is there because it makes a naive total wrong. The schema the models see, trimmed to the table that matters most: \sql CREATE TABLE deals (  id INTEGER PRIMARY KEY,  company\_id INTEGER REFERENCES companies(id),  owner\_id INTEGER REFERENCES users(id),  name TEXT,  pipeline TEXT,         -- 'new\_business' or 'renewal'  stage TEXT,            -- current stage: lead, qualified, proposal, negotiation, closed\_won, closed\_lost  amount REAL,           -- deal value in the deal's own currency  currency TEXT,         -- DKK, EUR, SEK or NOK  close\_date TEXT,       -- expected close date for open deals, actual close date for closed deals  archived INTEGER,      -- 1 = merged into another deal and kept for history only  merged\_into INTEGER    -- the deal this one was merged into, when archived = 1 ); \\ The other tables are deal\_contacts\ (deal, contact, role), deal\_line\_items\, stage\_history\ (deal, stage, entered_at), fx\_rates\ and users\. In the "without documentation" condition, the comments after --\ are stripped, so the model sees bare column names. ## Ground truth, computed twice truth.py\ computes the answer to every question twice: once in plain Python over the generated records, and once in SQL over the SQLite file. The script exits 1 if the two disagree by more than half a krone. For the CFO question, the SQL uses EXISTS\ so a deal with two CFOs counts once: \sql select sum(d.amount\*f.dkk\_per\_unit) from deals d join fx\_rates f on f.currency=d.currency where d.archived=0 and d.stage in ('lead','qualified','proposal','negotiation') and exists (select 1 from deal\_contacts c where c.deal\_id=d.id and c.role='CFO') \\ It also computes the naive answers, so you know what each trap would cost: DKK 12,527,857 instead of DKK 10,983,557 for the open pipeline without currency conversion (14% too big), DKK 11,772,774 with the archived copies counted (7%), and DKK 6,342,447 instead of DKK 5,083,337 when you join to contacts and sum (25%). ![Three horizontal bars on a light ground, one per trap. Each bar runs to a gold line at the true number and carries on in red: CFO rows counted twice gives 25 percent too much, currency not converted 14 percent, merged duplicates counted 7 percent](https://brinvik.com/claude-news/2026-11-09-claude-fable-pipeline/brinvik-pipeline-info-1-faelder-en-1600.jpg) ## The agent loop api\_job.py\ gives each model two tools: run\_sql\, which runs one SELECT against a read-only copy and returns at most 40 rows, and submit\_answer\, which takes the number, the SQL that produced it and a short explanation. The tool descriptions below are translated; the originals are in Danish, as is the question. \python TOOLS = \[    {"name": "run\_sql",     "description": "Run one SELECT against the CRM database (SQLite, read-only). "                    f"Returns at most {MAX\_ROWS} rows.",     "input\_schema": {"type": "object", "properties": {"sql": {"type": "string"}}, "required": \["sql"\]}},    {"name": "submit\_answer",     "description": "Submit your final answer: the number, the query that gave it and a short explanation.",     "input\_schema": {"type": "object", "properties": {         "value": {"type": "number"}, "sql": {"type": "string"}, "forklaring": {"type": "string"}},         "required": \["value", "sql", "forklaring"\]}}, \] \\ The loop is the plain tool-use loop: call the model, run every run\_sql\ it asked for, send the results back, and stop when submit\_answer\ arrives or after 14 turns. Errors are recorded, never hidden. Every one of the 96 runs ended with an answer. The connection opens the database with mode=ro\, so a model that tries an UPDATE gets an error back instead of a changed answer key. ## Scoring with a tolerance score.py\ counts an answer as right when it is within half a percent of the truth: \python right = abs(val - t) / abs(t) <= 0.005 \\ Half a percent absorbs rounding to whole kroner and tiny exchange-rate differences, and nothing else. No trap in this dataset is that small: the smallest, joining the stage history on question 5 so that a deal which entered proposal twice in October counts twice, is about 4%. The scorer also checks every wrong answer against the naive totals, so a miss is labelled with the trap that caused it, or with none. ## Cost per right answer, not per token Cost is input and output tokens times list price, with thinking counted as output. Per run, Fable 5.2 cost 18.0 cents, Opus 5.5 9.3 and Sonnet 5.5 2.7. Divide by right answers instead of runs and you get the number that matters: 2.7 cents per right answer for Sonnet, 9.3 for Opus and 19.9 for Fable, more than seven times Sonnet. Fable also used about twice Sonnet's output tokens on the same questions, and averaged 21 seconds per answer against 7. ![Three columns on a light ground, one per model, with 32 dots each. Fable 5.2 has 29 gold dots and three red rings, Opus 5.5 and Sonnet 5.5 have 32 gold dots. Under each: 19.9, 9.3 and 2.7 cents per right answer, with a bar showing the difference](https://brinvik.com/claude-news/2026-11-09-claude-fable-pipeline/brinvik-pipeline-info-2-pris-en-1600.jpg) ## What the results were No model summed a trap in any of the 96 runs. All three left out archived copies every time and converted currency in every run that asked for kroner. Sonnet and Opus matched the answer key 32/32. Fable matched it 29/32. Fable's three misses were all question 5, "the deals that went into proposal in October". It counted only deals whose first entry into proposal was in October (17 deals, DKK 2,797,423) where the key counts every entry in October (22 deals, DKK 3,563,279). None of them was a summing trap: the SQL was correct for the reading it chose. In all 4 of its runs on that question, Fable's explanation named both readings and both numbers. Opus did it in 1 of 4, Sonnet in none. Sonnet's full marks on question 5 came from its reading matching my key. ## Taking the documentation away The counterfactual was the part I was least sure of. With the column notes stripped, Opus and Sonnet answered the same, and Fable got one more right (15/16 against 14/16), so the documentation made no difference you could measure here. Mean queries per answer went up a little: Sonnet 3.9 to 4.1, Opus 4.9 to 5.5, Fable 5.6 to 6.3. The models found archived\, the currency column and the contacts table by querying the data. The caveat is that the names were clear; a column called status\_2\ would be a different test. ## Running it without an API key in the repo The calls run in a GitHub Actions workflow that authenticates with workload identity federation, so no key exists anywhere: the client takes its identity from the workflow's federation settings, and the job only runs on the main branch by manual dispatch. verify.py\ then rebuilds the database from pack.py\, checks that the recorded copy matches it row for row, re-reads every run and writes the facts file every number in the write-up comes from. Doc set and bare set cost $4.72 and $4.86. ## Claude for business in practice: doing this on your own CRM 1. Pick five to ten questions you already know the answer to, and write the true number down before you start. 2. Give each model the schema, one read-only query tool and a submit tool that asks for the number, the SQL and an explanation. 3. Score with a tolerance, and label each miss with the trap it fell into, if any. 4. Divide cost by right answers. 5. Read the explanations, not just the numbers. That is where the ambiguous question showed up. Anthropic's own advice is to start with Opus 5.5 and move up or down as needed. On these eight questions Sonnet was as accurate as Opus at under a third of the price, so I would include Sonnet in your test; that is the advice of this one test, on invented data. The full write-up, with a demo of every run and the SQL each model wrote, is [the article testing Claude Fable 5.2 on a sales pipeline](https://brinvik.com/en/journal/claude-fable-5-2-pricing-sales-pipeline-test?utm_source=devto&utm_medium=article&utm_campaign=2026-11-09-claude-fable-pipeline&utm_content=devto-en). There is a [Danish version of the pipeline test](https://brinvik.com/da/journal/claude-pris-fable-5-2-salgspipeline-test?utm_source=devto&utm_medium=article&utm_campaign=2026-11-09-claude-fable-pipeline&utm_content=devto-da) too. If you want this set up against your own CRM, that is part of my [AI advisory work for sales and RevOps teams](https://brinvik.com/en/services/ai-advisory?utm_source=devto&utm_medium=article&utm_campaign=2026-11-09-claude-fable-pipeline&utm_content=devto-service).