13 papers on AI agent memory: the biggest lesson is to keep the originals

Why I read them

I'm building Folio Continuity, a shared project memory that AI agents read from and write to. One design question kept coming up: should an agent distill a project's history into one tidy page, and keep rewriting that page as work goes on?

So I went through 13 recent papers on agent memory. With help from AI agents, we read the full text rather than just the abstracts, and checked the key numbers against the papers. That turned out to matter, as you'll see below.

Here's my short version:

Rewriting memory over and over loses facts.
Keep the original records. Treat the distilled version as a shortcut you can rebuild.
Distill less often, and on purpose.
Handle must-keep rules separately from general notes.
Anything an agent reads can end up in its memory, so track where each memory came from.
Don't compare benchmark scores across papers.

Lesson 1: Rewriting memory loses facts

The clearest result is from Useful Memories Become Faulty When Continuously Updated by LLMs. On a stream of synthetic puzzles, GPT-5.4 solved this share of new puzzles, averaged over 10 runs:

Memory	New puzzles solved
None	38.6%
A summary rewritten after every puzzle	43.2%
Raw records of past puzzles, never consolidated	70.4%

The rewritten memory barely beat having none. The paper also reports that after building memory from the correct solutions to puzzles the agent had already solved, it then failed 46% of them.

FactRot finds the same pattern in plain text. After an LLM rewrote Wikipedia passages 12 times (8 passages, four models), "paraphrase faithfully" kept 75% of the tracked facts, while "answer, then restate" kept 47%.

Lesson 2: Keep the originals

RecMem stores every turn word for word in a cheap raw layer and distills only later. Remove that raw layer, and its score drops from 81.1 to 51.9.

In What to Keep, What to Forget, keeping the originals and searching them held recall near 95%, against 33–56% for repeated summarizing. That was a small test with one small model (Qwen2.5-1.5B), but it points the same way.

My take: treat the distilled page as an index, not the record. When it needs updating, rebuild it from the originals rather than editing it from itself.

Lesson 3: Distill less often, and on purpose

My take: distilling is still worth doing; it just shouldn't happen on every update. Two papers point that way:

RecMem calls the LLM only when similar interactions come up again, which cut memory-building tokens by up to 87% compared with three existing memory systems.
Nemori distills new facts from what its existing memory failed to predict. On LoCoMo with gpt-4o-mini, it scored 73.0 against 72.3 for reading the whole chat, using 2,745 tokens per answer instead of 23,653.

Lesson 4: Protect must-keep rules separately

The Compaction Cliff asked Sonnet 4.6 to compress agent configuration files while keeping every safety rule word for word. 53% of the rules survived one round, and 10% survived five. A method that labels each line and copies rules through unchanged kept 96%.

Two caveats from reading the paper and its released code. The budget halves every round, and "kept" is a keyword check. And the code calls Sonnet 4.6 through the Agent SDK with a compression prompt written in the repo, rather than running Claude Code's /compact command as the paper's wording suggests.

My take: the shrinking budget forces some of that loss, but the direction holds. Rules shouldn't go through the same summarizer as everything else.

AgingBench reports a quieter version of this: behavioural tests can stay clean while factual precision decays. In one setup, precision fell from 0.90 to 0.37.

Lesson 5: Anything an agent reads can become memory

From Untrusted Input to Trusted Memory ran 3,240 attack cases against two agent frameworks. Harmful instructions hidden in ordinary content got written to memory 34% of the time on OpenClaw and 67% on HERMES. Prompt-injection detectors caught at most 47% of the subtler "weak-signal" attacks.

Being written to memory isn't the same as causing harm, and the paper measures that separately.

My take: memory is an attack surface. Record where each memory came from, and don't treat saved memory as trusted by default.

Lesson 6: Benchmark scores don't travel

When Does Belief-Based Agent Memory Help? graded the same answers two ways. Word-overlap F1 gave 32.45, and an LLM judge gave 59.97.

Setups differ across papers too. The same "give the model the whole conversation" baseline, with the same model (gpt-4.1-mini), scored 80.6 on the LoCoMo benchmark in one paper and 91.3 in another.

My take: only compare results run in the same setup.

What we changed

In Folio Continuity, a person's agent saves a working session into a project: a session note, a log line, the files worth keeping, and a one-page memory. Much of that already fit the research. Nothing saves without the person's yes, the log only ever adds lines, and the files they choose to keep are stored as they are.

One part didn't fit. On every save, the agent writes a whole new memory page based on the last one, and the person's yes only covered a short summary of the session. So they never saw what the page lost.

Now our instructions ask the agent to quote, in the preview, every line the save would remove or reword, and to list what it adds. Client rules stay word for word unless the person agrees to change them, and every new line names its source. This is guidance to the agent, not something code enforces, so the next step is measuring whether agents follow it.

We applied the same thinking to our own agents' notes:

History: every note is now committed to a local git repo at the end of each session, so any edit can be undone.
Index size: we watch the size of the memory index, because Claude Code loads only the first 200 lines or 25 KB of MEMORY.md. New lines go at the bottom, so the newest are the first to silently stop loading.

Caveats, and why to read past the abstract

Most of these papers are preprints, many tests are synthetic, and we didn't find independent replications.

My take: trust the direction more than any single number.

Read the full text. One abstract says the agent "fails on 54%" of puzzles it had solved, while the body says 46%. A summary that an AI wrote for me also misread a table.

That's the same lesson as the papers: check the summary against the source.

If you're building agent memory, I'd like to hear what you've seen. Do you let agents rewrite their memory, and how do you check what they lose?