How AI Agent Memory Architectures Split Working Memory from Episodic Storage

Your agent remembers nothing between sessions. Not "forgets a little" — nothing. The model is stateless, and every "memory" your agent has is tokens you shoved into its context window. The real question isn't whether to add memory. It's where each kind of memory lives — and most teams get this wrong by treating the context window like a database.

The context window is RAM, not a hard drive

Working memory is everything the model can see in one inference: the system prompt, the conversation, tool results, the reasoning trace. It vanishes when the run ends. And here's the part vendors won't put on the slide: bigger windows didn't fix this. GPT-5 takes up to 272,000 input tokens, and OpenAI's own cookbook warns that even that can be overwhelmed by uncurated histories. Anthropic calls it context rot — recall gets worse as the window fills, because every token attends to every other token. Chroma tested 18 models with only input length varied: performance degraded in every single experiment. Stuffing more in doesn't make the agent smarter. It makes it noisier.

Trimming vs. summarization: both are a tax

You have two ways to manage the pressure, and neither is free. Trimming drops old turns — cheap, deterministic, and it forgets abruptly. Summarization compresses history into injected summaries — it keeps long-range context, but adds latency and risks drift. Pick your poison; there is no free lunch here, only a choice about which failure mode you can live with.

Episodic storage is the layer most teams skip — and then regret

Episodic memory is the durable record: a specific interaction, at a specific time, with a specific outcome. It lives outside the window in an external store and gets retrieved on demand, usually by semantic search. Mem0 formalises the pattern — extract durable facts, deduplicate, embed, save, retrieve by combined semantic, keyword, entity and temporal score. LangGraph splits it cleanly: a checkpointer holds history within one thread, a BaseStore holds facts across all sessions. The store should match how you actually recall: vector search for similarity, timestamped logs for recency, a structured database for exact lookup. Picking a vector store because it's fashionable, when your recall pattern is "what happened last Tuesday," is architecture by trend-chasing.

MemGPT had the right mental model in 2023 — and it still holds

The Berkeley paper framed agent memory as an OS problem: context window as RAM, external databases as disk, and the model paging information in and out with function calls. When the agent wants a past episode, it asks for it — self-directed paging, not the application stuffing the prompt in advance. On its long multi-session test, GPT-4's accuracy jumped from 32.1% to 92.5% with paging. The honest caveat: the follow-up analysis found the other benchmark small and retrieval-focused, and paging matters less when the window can hold the history. It helps enormously exactly when the window can't — which is the case that actually bites in production.

The three crossings — and the one everyone forgets

Information moves between layers on three paths: the write path (what gets promoted from working memory into storage, and when), the recall path (what gets pulled back, and how much), and consolidation — reading accumulated episodes and distilling them into summaries, preferences, lessons learned. Consolidation is the one most often skipped, and it's the one that keeps the store from bloating into noise. MemGPT warns the model at 70% of the window to move useful information out. Anthropic's Claude Plays Pokemon agent kept strategic notes across thousands of steps and survived context resets. Without consolidation, a year of episodes is just a junk drawer with a search bar.

The verdict

Working memory alone is fine for single-session, short tasks — most chatbots live here comfortably. But the moment a returning user shouldn't have to repeat themselves, you need a store outside the window. And the store is only as good as the discipline around it: a write path that fires, a recall path that retrieves selectively instead of drowning the context, and consolidation that keeps the record honest. Memory isn't a feature you bolt on. It's the architecture that decides whether your agent learns — or just re-reads its own transcript forever.