Part 8 of ten. Each post is a short, plain-words tour of one lesson from e=mc², a free build-an-LLM-from-scratch course; the full lesson, notebook and numbers are on the site.

A language model that only talks is a toy; the moment it has to fill in a form that a program will run, it becomes a tool, and that is exactly where it starts to break. Yesterday's post left you a tiny model that could read a question and name the right operation to run. That is half a decision. Today the same model fills in the other half, which metric, which territory, which month, how many, as JSON a program can execute with nobody in between. I want to show you the one trick that makes every answer valid, and the one number that trick cannot touch.

The form at the counter

Picture Momo, the monkey, working at a message post with seven counters. Each counter does one job, and each needs its own form filled in first. One counter ranks a metric; its form wants a metric from a list of ten, a level, and optionally a count, a direction, a territory code like T-123, a product and a month. Each box on the form is a slot. The filled-in form is JSON: a short text of keys and values, the way programs like to be told things.

Momo has 0.84 million parameters, small enough to answer in six milliseconds on an ordinary CPU, and small enough to make small mistakes. Writing freely, he gets a lot right and then adds a box the form does not have, or forgets to close the final brace. The counter rejects the whole form.

Ella's stencil

Ella, the elephant, is the teacher. Tonight she holds a stencil over Momo's pen.

This is constrained decoding. A model writes one token at a time, with a scored list of guesses before each. Ella looks at what Momo has written so far and asks of every possible next token: could this still be finished into a form the counter accepts? Every token that could not is masked out before Momo picks. He still chooses, but only among tokens that keep the form alive.

The stencil is a grammar, and the question it asks is called prefix validity. Nobody hand-wrote this grammar; it is built from the catalog that the real project, pankhllm, already keeps: which operations exist, which parameters each takes and in what order, which words each pick-list allows, what shape a code or a month must have, what range a number must fall in. Momo cannot write a parameter the operation lacks, skip a required one, or close the brace too early.

One lovely detail. Momo has a vocabulary of words, so he cannot reliably spell a code like T-412. Instead he writes a pointer, in effect "the first code in the question", and Ella copies the real characters in afterwards.

The whole stencil is a Python function called once per token. Condensed, it fits on a card:

# condensed from the notebook's value_options and decode def value_options(spec, ctx): # legal tokens after a parameter name if spec["type"] == "enum": # listed words (grounded: only ones the question names) return {f'"{v}"' for v in spec["values"] if not ctx["grounded"] or named_in(ctx["q"], v, spec)} if spec["type"] == "string": # markers whose real value fits the pattern return {m for m, v in ctx["vals"].items() if re.fullmatch(spec["pattern"], v)} if spec["type"] == "integer": # number markers inside the range return {m for m, v in ctx["vals"].items() if v.isdigit() and spec["min"] <= int(v) <= spec["max"]} logits = logits + mask(allowed_next(ctx, written_so_far)) # -inf everywhere else, then argmax 

On the site, a widget steps through it: three questions, token by token, Momo's top guesses on one side and what the stencil allowed on the other.

Two stencils, one extra rule

The lesson builds two. The catalog stencil enforces everything the catalog says about shape. The grounded stencil adds one rule: a pick-list word is allowed only if that word, or a known alias of it, appears in the question. If nobody mentioned a product, Momo cannot add one. The aliases come from the catalog too; "total scripts" means one metric, "YoY" means a comparison to the prior year.

That one rule turns out to be the whole story.

The numbers, honestly

One run, 11 October 2026, on an Apple Silicon Mac with the GPU shared with other jobs; the whole notebook took 96.5 seconds. The test is 2,000 questions from templates the model never trained on, always taking the top token.

  • Free decoding, no stencil: 91.6% valid JSON. 47.3% exactly right, operation and every parameter. 44.4% valid but wrong. Operation alone right 84.5% of the time. The failures were near-misses: an extra parameter on an operation that has none, and 43 outputs that did not even parse.
  • Catalog stencil: validity 100.0%. Exact match barely moved, 47.3% to 50.5%, and the operation was right 84.5% of the time, exactly as before. The invalid answers did not become right; they became valid wrong ones, 49.5%.
  • Grounded stencil: validity 100.0%, exact match 71.3%, wrong but valid 28.7%, operation right 90.8%. Momo could no longer name a product nobody asked about.
  • Speed: a full JSON answer took 6.0 milliseconds at the median on the CPU, about 11 tokens. The operation alone, 0.87 milliseconds.

Of the 574 wrong-but-valid answers left, 390 had the right operation with one parameter wrong or missing, often a dropped "bottom", and 184 had the wrong operation outright.

The error a grammar cannot fix

Here is the idea I most want you to leave with. A grammar guarantees the format and nothing about the meaning. The catalog stencil proves it: validity jumped to 100% and the number of right answers hardly changed, because every answer that used to fail loudly now failed quietly, as a perfectly shaped form filled in wrongly.

That quiet failure is the dangerous one. A program will happily run a valid, wrong request. So the number to watch is wrong but valid, not "how often is the output valid". Grounding helped because it is the one part of the stencil that checks the answer against the question: a check on meaning, not on shape. Hosted "JSON mode" features in big APIs are the same idea with a bigger grammar, and the same limit.

The confidence score carries the same lesson. With the grounded stencil, a fixed threshold let the model decide 72.1% of the questions at 98.54% precision. An earlier run of the same notebook gave 93.49% at that threshold, so I do not lean on it. On 14 questions written for a different catalog, the operation was right on 11, and one miss was the bad kind:

Why did calls fall in T-112 in 2026-08?

The model sent that to the counter that reports a KPI's value, with a confidence of 0.996. Valid, confident, and wrong. The next lesson replaces that confidence with something sturdier.

What you can take from this

If you ever ask a model for JSON, do two things. Constrain the decoding; it is a short function and it turns the format from a hope into a guarantee. Then measure the error that still parses, because that is the one the grammar hides. The three-way results table, the condensed stencil and the notebook that rebuilds all 22,000 questions byte for byte before training are all on the site.

Want to actually build it?

This post is the postcard. The lesson is the trip. Lesson 08 of e=mc² on theaivibe.org walks you through every piece with widgets you can poke, a notebook that runs on a free GPU in minutes, and the honest numbers at every step. No account, no paywall, nothing to install.

Tomorrow, Lesson 09 leaves templates behind: a bigger model labels real human phrasings, and three small models, today's among them, learn from 183 of its own decisions.