Agent Memory Is Not a Junk Drawer: What Anthropic's Dreaming Experiments Teach About Building It

Most agent memory I see in production is a junk drawer: every tool result, every user message, every "fact" the agent thought was interesting, appended to a vector store and retrieved by cosine similarity forever.

Anthropic just published a research post, The components of agent memory (Nov 6, 2026, by Lance Martin), that argues for something more deliberate. Here is what they tested, what the numbers say, and how I'd apply it to an agent you're building today.

What "agent memory" means here

Anthropic's definition is narrow on purpose: knowledge an agent gains from experience (users, tasks) and keeps between sessions, saved as context it can read later. It is not weight updates, not in-context learning inside one chat, and not compaction (summaries that replace older turns).

They describe memory as layers:

Transcripts: the raw session logs. The only ground truth.
Notes: rules, verified facts and traps the agent writes while it works.
Representations: linked wiki pages that map what the agent knows.
Injected context: a small layer the harness loads at the start of every session.

And a process beside them called dreaming: offline passes (building on the sleep-time compute idea from Letta and UC Berkeley) where the agent rereads transcripts to correct notes, build wikis, fill gaps and decide what gets promoted to the top layer.

Finding 1: saving everything makes memory worse

The clearest example is Claude playing Pokémon Red, compared about 10,000 steps into the game:

Claude Sonnet 3.7 wrote to memory indiscriminately: 413 memory files across 225 folders, one per battle, item or character. It earned 1 badge. At one point it spent 17 steps creating six files and deleting nine others while taking no actions in the game.
Claude Fable 5 wrote 9 files in two folders plus one global.md (current goal, a plan for gyms, its own memory rules). It earned 4 badges.

The difference is what each model considered worth remembering. Money and map position can be read off the screen every turn, so storing them is redundant and goes stale. Strategies, plans and tested rules exist only if the agent works them out and writes them down.

Lesson: don't store what the environment already tells you. Store what the agent learned.

Finding 2: tell the model what memory is for, not how to use it

On Continual Learning Bench (CL-Bench) 1.0, a database question task where the agent gets the correct answer after each attempt and can write memory for later sessions:

Opus 5 was the first model to "close the loop": entries with a rule, how it knew the rule was true, earlier answers and warnings about traps.
Telling Opus 5 and Opus 4.7 to "record only failures" made them lose much of the gain memory gave them. For stronger models, freedom to judge salience beat fixed rules.
Older models needed steering. Prompting Sonnet 4.6 to read memory before answering and test its theories nearly doubled its score. Sonnet 5 read memory before answering 9 out of 10 times on its own; Sonnet 4.6 did so less than half the time.

Lesson: with a strong model, a short "here is a memory folder, here is what it's for" beats a rulebook. With a weaker model, add explicit "read memory first, test your theory" steps.

Finding 3: notes go wrong, and transcripts can fix them

In the Pokémon run, Claude wrote down (wrongly) that the PokéCenter was on the east side of Mt. Moon, declared "Mt Moon DONE", walked off a one-way ledge and kept looping back to the previous route. Without a fix, it never reached Cerulean City.

A dreaming pass checked that note against the session transcript, caught the error and rewrote it. With the corrected note, Claude went back into the cave and reached Cerulean City.

Lesson: a memory note is a claim. Keep the raw transcript so something can audit the claim later.

Finding 4: wikis help, but only for cross-document questions

Anthropic built wikis with dreaming over a corpus of about 350,000 files (~420M tokens) from Stainless, with 25 evaluation questions:

Scaling dreaming from 2 to 128 sessions raised the wiki-only score from 13% to 48% of graded criteria.
Search + wiki rose from 54% to 61%, versus 53% with search alone, and saved about 20,000 tokens per question.

On 24 merger questions where more than half needed info from several documents, the wiki-based answers scored 87% vs 73% for search alone on those multi-document items. But on a Harvey LAB legal brief task where most criteria came from a single document, the wiki showed no clear gain. On another task, the wiki grew larger than the source documents: a map as big as the territory.

Lesson: build linked representations when answers need joining facts across sources. If one text search finds the answer, a wiki is just maintenance cost.

Finding 5: the instructions for building memory matter

On one LAB acquisition task, several agents wrote contract terms into one shared table at the same time and overwrote each other. The brief built from that wiki included only 41 of 57 graded facts (72%).

After a reflect-and-rewrite loop on the instructions, each contract got its own page, written by one agent, quoting terms word for word. Result: 57 of 57 facts (100%).

Lesson: tell the memory builder what actually matters for the downstream task, and avoid many writers on one shared page.

Finding 6: keep the injected layer small

Promoted notes are loaded into every session, which costs tokens and carries any wrong note everywhere. Claude Code's auto memory loads only the first 200 lines or 25 KB of MEMORY.md, used as an index of what it knows and where to find it. Anthropic's docs also warn that longer instruction files reduce adherence.

Lesson: the always-loaded layer is an index, not an encyclopedia. Push detail down and leave a link.

A practical design for your own agent

Here's how I'd translate this into an agent you can ship (my synthesis, not Anthropic's code):

Log full transcripts with run IDs. They're your ground truth and cheap to store.
Give the agent one memory folder and a short instruction about what memory is for: rules it verified, plans, traps. Not environment state it can re-read.
Store provenance with each note: "rule X, confirmed by grader on run 123". An unverified note should say so.
Run a scheduled dreaming job (nightly or after N sessions) that:
- checks notes against the transcripts they came from and fixes or deletes wrong ones,
- merges duplicates and repairs links to missing pages,
- promotes only a handful of high-value notes to the injected index.
Cap the injected layer (lines or bytes) and enforce it in code.
Build a wiki only where questions span documents, and measure it against plain search on your own eval set before keeping it.
Focus dreaming on what changes. Anthropic notes that continuously checking every memory isn't practical; prioritize volatile information or tune cadence by how often memories are accessed.

Open questions they flag

Writing memory in-session captures salience at the source but pulls attention from the task; dreaming moves it to the background but its output isn't immediately available.
Dreaming costs compute, and maintained representations can accumulate errors or hallucinations.
Transcript search alone may be a reasonable foundation, with representations added on top only where they earn it.