The most useful number published in AI this month did not come from a model launch. It came from a data company.
On a recent Latent Space deep dive, Garrett Lord — founder of Handshake AI, whose training-data business reportedly reached roughly $1 billion in gross annualized revenue in about a year — walked through what frontier labs actually buy from him. The context for the whole market is uncomfortable: on Handshake's finance benchmark Atlas, where the task is building the kind of financial model a junior banker delivers, frontier models score below 20%. On Vials, its life-sciences benchmark, most models stay under 50% even when given tools.
Lord put the market for AI training data at roughly $10 billion today, up from about $1 billion three years ago, and forecast $100 billion within three years. Remember who is forecasting. But the direction is what matters: the money is moving away from raw data and toward something else.
What the money actually buys now
Handshake's product is not rows of labeled text. The company builds simulated work environments — a fake investment bank with email threads, a data room, financial files that tie out — then has domain experts author tasks and grading rubrics inside them. Lord's line on the labor: "labeling" is the wrong verb now; "it's experts authoring data." His line on the shortcut: if you just ask a model to generate the environment itself, "it's slop, it doesn't work."
Two details from the episode are worth sitting with:
- The labor has shifted from students at roughly $10 an hour drawing boxes around cars to professionals authoring environments. The company runs about 800 people and expects that to become thousands.
- One of Handshake's finance leads noted that labs may use under 10% of the data they buy — because the mix matters as much as the quality of any single item.
Read that as a small-business owner and the signal is blunt. The expensive input in AI right now is not compute, not the newest model, and not prompt tricks. It is real examples of finished work — inputs with known right answers — plus the expert judgment about what makes them right.
Your business already produces this asset
Every small business produces exactly what labs are paying for, and throws most of it away. The quote that won the job and the quote that lost it. The email thread where you talked a customer down. Last month's invoices. The correction you made to an AI draft this morning — the one that exists nowhere except your head.
That is expert-authored data. It evaporates because nobody files it: quotes go out, replies arrive, the folder moves on. The lab version of that asset takes days per item to produce. Your version already exists, and collecting it costs an hour.
The 20-minute answer-key file
Here is the kitchen-table version of what Handshake sells:
- Pick one recurring job family — the one where AI already drafts work for you (or should). Quote replies, weekly reports, customer responses. One family, not the whole business.
- Collect five real examples. Each one is the input you received and your actual finished output. De-identify anything sensitive. Five is enough to start.
- Write one line per example: what makes this one right — the thing an outsider could not see. That line is the judgment, and it is the part you cannot buy.
- Use it two ways. To grade: before any AI draft for that job leaves your hands, check it against the five examples. To guide: paste the closest example into the prompt when you delegate the job — a real finished example teaches a model more than a paragraph of adjectives.
- Keep it private and keep it current. Labs on the episode explained why they hold evaluation tasks back: a public test leaks into training sets and stops testing anything. The same logic protects your file — its value is that it is specific to your business and unseen outside it. Refresh it when the work changes, not on a calendar.
The correction log is the second half of the asset
Second practice, five minutes a day: every time you fix an AI draft, write down what was wrong and the rule it broke. Two columns, one running file. "Used the wrong payment terms — always check terms against the signed contract." "Invented a delivery date — never state a date unless it came from dispatch."
Labs pay professionals to author exactly this kind of signal. You generate it for free every time you correct a draft — and almost always let it evaporate. Ten entries in, the log becomes part of the prompt for the next job: the same recurring failure patterns, now pre-answered. That is the compounding half of the answer-key file: examples for the model to imitate, corrections for it to avoid.
Where to keep the scale honest
- Sub-20% is a lab-built benchmark for investment-bank work. It says closed-domain expert work is still hard for models — a direction, not a prediction about your quote replies.
- The $100 billion figure is a forecast from someone selling the data. Treat it as a data point with a visible thumb on the scale.
- Handshake's machinery — 800 people, simulated banks, rubrics of roughly 146 criteria — is lab machinery. You do not need it. The five-example file and the correction log are the scale-honest translation, and they are a lab practice transcribed, not a small-business-benchmarked result.
- Models change under you. A file that graded clean last quarter is a snapshot; rerun your five examples when your AI vendor ships a major update.
The market is moving toward paying for judgment-shaped data. That is the whole signal: the scarce input is not a bigger model — it is real examples and the judgment about what makes them right. A small business generates both every week. The only question is whether they get filed.
If you want the systems side of this — which workflows are worth handing to agents in the first place, and how to wire them so review stays cheap — the AI Operations Playbook covers agent blueprints for support, content, sales, research, finance, and project management, with review checkpoints built into each one.
