Part 3 of ten. Each post is a short, plain-words tour of one lesson from e=mc², a free build-an-LLM-from-scratch course; the full lesson, notebook and numbers are on the site.
The same little language model, trained on the same stories for the same 1,500 steps, either makes things up or recites, and the only thing that decides which is how many dials it has. In Lesson 01 I built a 5.9-million-parameter storyteller, Momo, and caught him reciting 61 words of a training story word for word. That left a question one model cannot answer. How small does Momo have to be before he stops reciting and starts composing? So in Lesson 03 I built him four times.
Four Momos, one Ella
A quick refresher. Momo is a young monkey learning to tell stories; inside his head are dials, and the dials are the model's parameters. Ella is an elephant who never forgets. She reads Momo stories one word at a time, lets him guess the next word, tells him how wrong he was, and nudges his dials. That nudge is training.
Today's twist is that there are four Momos. One has 953,856 dials; call him the 1M model. The next has 3,060,288 (3M), then 5,853,184 (6M, which is exactly Lesson 01's Momo), then 12,318,720 (12M). Everything else is held fixed on purpose: the same 22,800 training stories, the same tokenizer, the same 1,500 steps, the same random seed, so all four even see the same batches in the same order. Size is the only thing that differs.
Only two numbers change between them: how many blocks are stacked, and how wide each token's list of numbers is. That is the whole experiment.
SIZES = { # name -> (n_layer, n_embd); 4 heads, 256-token window, dropout 0 in all "1m": (2, 128), "3m": (5, 192), "6m": (6, 256), # Lesson 01's config "12m": (6, 384), } # per size: seed 42, build GPT, seed 42 again (same batches), 1,500 steps, then measure for name, (nl, ne) in SIZES.items(): model, cfg, r = train_one(name, nl, ne) r.update(recite(model)) # 50 prompts x 120 tokens, temperature 0.8, top-k 40, seed 1234 The stories Ella reads three times
The training set is mostly TinyStories, 19,800 short children's stories. Mixed in are 1,000 Momo-and-Ella stories I wrote, and Ella reads each of those three times. Another 200 TinyStories are kept back as a quiz Momo never sees, with no Momo in them at all.
Those thousand repeated stories are the trap. If you are a pile of dials trying to be less wrong about the next word, and the same story comes round three times, the cheapest trick is to simply remember it. The question is how many dials it takes before that trick becomes available.
How you catch a monkey reciting
You cannot ask Momo whether he is copying, so I measure. Each model gets the first ten tokens of 50 training Momo stories and writes 120 more tokens. Then I take what he wrote and find the longest stretch of it that appears, character for character, inside any of the 1,000 training Momo stories. I call that stretch the run.
A run of 100 characters is about three of the short Momo sentences (they average 32 characters), or roughly a quarter of an average 417-character story. If your median run is past 100, you are not composing. You are reciting.
The numbers, honestly
Here is what the four sizes did, smallest to largest.
- Quiz perplexity: 16.0, 10.8, 9.0, 7.4. Perplexity is "between how many words is Momo still choosing" on the 200 stories he never saw. Bigger Momo, better guesses, as you would expect. The 6M row lands at 9.0, Lesson 01's figure.
- Loss on the Momo stories themselves: 0.554, 0.247, 0.206, 0.185. This is how surprised each model is by the stories it trained on. The drop between the first two sizes is the big one.
- Median longest copied run: 62, 211.5, 265 and 269 characters.
- Generations with a run of 100 or more: 6% (3 of 50), 98% (49 of 50), 100%, 100%.
Read the last two lines together. The 1M model mostly composes. Its median run is 62 characters, and only 3 of its 50 stories contain a 100-character copy. Its stories wander and mix sentences into things like this:
tell a soft pink banana that
Then at 3.06 million parameters it flips. Forty-nine of fifty generations contain a copied run of 100 characters or more. The median is 211.5 characters and the longest is 361, most of a whole story. At 6M and 12M every single generation passes 100, and the first sample from both of those models is the very same 266-character paragraph, which turns out to occur word for word in three different training stories.
The check number is 3M. The smallest size whose median copied run crosses 100 characters is the 3,060,288-parameter model. Somewhere between 0.95M and 3.06M dials, recitation switched on.
The part that surprised me
I expected memorisation to show up the way the textbooks say it does: training loss keeps falling, quiz loss starts climbing, a gap opens. It did not. At every size, train loss sat above quiz loss, the opposite of an overfitting gap. If I had only looked at those two curves I would have said the larger models were fine. They were reciting the whole time.
That is the one idea I want you to carry away from this post. Memorising a repeated corner of the data need not show up in the headline numbers at all. The Momo-story loss and the copied-run table are where it shows, and you only see it if you go looking.
What this does not prove
I owe you the disappointing parts too. This is Momo's friendliest possible case for memorising: the thousand Momo stories are built from small pools of stock sentences and read three times each. My run metric cannot tell one story learned by heart from a string of stock sentences that many stories share, so treat "copied run" as how much of the output is lifted from the training set, not as proof of story-by-story recall. I trained four sizes, so I can only say the switch happened between 0.95M and 3.06M parameters, not where. The 6M and 12M medians differ by 4 characters, which is noise. One seed and 50 prompts means no error bars. And none of this predicts where the thresholds sit for GPT-scale models; the research on those uses far larger models and far messier data.
Run it yourself
All four train in minutes. On an Apple Silicon Mac this run took 0.55, 1.70, 3.79 and 6.33 minutes for the four sizes, and the whole notebook about 10 to 13 minutes. It also runs on a free Colab T4, though I have not timed it there. The full lesson on the site has the glossary, both tables, the short trick that finds the copied runs, and three check-yourself questions, including the one I am still chewing on: how would these numbers change if the thousand Momo stories were genuinely different from one another?
Want to actually build it?
This post is the postcard. The lesson is the trip. Lesson 03 of e=mc² on theaivibe.org walks you through every piece with widgets you can poke, a notebook that runs on a free GPU in minutes, and the honest numbers at every step. No account, no paywall, nothing to install.
- Start here: https://theaivibe.org/emc2/why-small-llms-memorize-train-four-sizes
- The whole ten-lesson journey: https://theaivibe.org/emc2
Tomorrow: Lesson 04 takes the public Lesson 01 weights, teaches Momo a brand-new character, and measures exactly what he forgets along the way.