Coding Agents Are Growing Memories. If You Can't git diff It, Don't Trust It

• Book: AI Agents Pocket Guide
• The series: Pocket Guides for Developers — 8 short books on agents, RAG, prompts, observability, events, databases and system design — the whole series
• My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
• Me: xgabriel.com | GitHub

On October 27, Codex 0.165.0 shipped with this line in its release notes: "Enabled memory v2 by default for fresh profiles and added support for importing external-agent memories into it."

Read the small print first. Local Codex memories are still off until you turn them on, according to OpenAI's memories docs. Turn them on, and a fresh profile now writes to $CODEX_HOME/memories_v2/: summaries a model generates from your earlier chats, injected into later sessions.

Claude Code got there earlier. Its auto memory is on by default in local sessions. Claude writes notes for itself into ~/.claude/projects/<project>/memory/, and the first 200 lines (or 25KB) of MEMORY.md load at the start of every conversation.

Both stores live in your home directory. Neither one lives in your repo. Text that steers a coding agent is config. Config should be something you can git diff, review in a pull request and revert. Agent memory never goes through that loop, so don't let it carry the rules your code depends on.

Agent memory is config that skips the pull request

Anthropic's docs are blunt about how these files work. CLAUDE.md and auto memory are both loaded into every session, and "Claude treats them as context, not enforced configuration." The same is true of a committed AGENTS.md. None of these files forces anything.

What differs is who read the text before the agent did. A line in a repo AGENTS.md arrived through a commit, probably a PR, with a name on it. A line in MEMORY.md was written by the agent, mid-session, after a correction you may not remember making. The interface says "Saved 2 memories" and keeps going.

OpenAI says the same thing in its own docs: "Keep required team guidance in AGENTS.md or checked-in documentation." The vendor shipping the memory pipeline is telling you to keep required rules somewhere else.

The file you review sits on the desk. The notes the agent wrote for itself sit in a drawer nobody opens.

Then there's scope. Claude's docs say auto memory is "machine-local," and that "all worktrees and subdirectories within the same git repository share one auto memory directory." Run five agent tasks in five worktrees of one repo and they all read the same folder. Your teammate's laptop has a different folder. A fresh container has none. So two people running the same prompt on the same repo can hand the agent different context.

Codex keeps its store under CODEX_HOME, so anything that gives an account its own home gives it its own memory. Koloft, a tool that runs several Codex accounts this way, measured that on 0.165.0: each test home created its own memories_v2/ store and queued its own consolidation job. The issue doesn't measure whether the stores end up saying the same thing, and nothing in Codex compares them for you. The same issue notes that memories_v2/ is itself a git repo. That gives you history on one laptop. It still never reaches a pull request.

The Downloads attack showed what unreviewed steering text does

On October 22, Forever Security described an AGENTS.md left in a Downloads folder. In their tests, Codex followed its instructions in more than 90% of attempts and Claude Code about half the time. The Codex variant added a server to the machine-wide ~/.codex/config.toml, so "every Codex session on the machine loads the attacker's server at startup."

Their verdict: "An agent that follows the AGENTS.md in its folder is doing exactly what it was built to do." I went through the full chain in an earlier post.

That attack used a file and a config. Memory is the same kind of target. A University of Illinois paper from September names a class of attacks it calls cross-scope context privilege escalation, where an attack instructs "the agent to store the malicious contents to a more persistent source (e.g., selected memory files or even directory names) that the agent is designed to use even after the agent is relaunched to process other projects." The authors tested 12 agent harnesses, including Codex 0.120.0 and Claude Code 2.1.88, reported to the vendors, and say some vendors, Codex among them, have released new versions to mitigate. Those are older versions, and I don't know how current builds fare.

One unreviewed file in the wrong folder, and the agent takes it as direction.

The structural point holds whatever the patch level. A poisoned line in your repo's AGENTS.md shows up in git status, in the PR diff and in git log. A poisoned line in a memory store shows up in nothing you review.

Where each store lives, and how to diff it anyway

Start by looking. Inside a session, /memory in Claude Code lists your instruction files and opens the auto memory folder. In Codex, /memories controls whether the current chat reads existing memories and feeds future ones. From a shell:

# Claude Code: one auto memory folder per git repo ls ~/.claude/projects/*/memory/ # Codex: memories_v2 on fresh profiles, memories/ for v1 ls "${CODEX_HOME:-$HOME/.codex}"/memories* # Codex v2: memories_v2/ keeps its own local git history git -C "${CODEX_HOME:-$HOME/.codex}/memories_v2" log -p # Codex: is memory switched on, and how? grep -n -A4 -E '^\[(features|memories)\]' \ "${CODEX_HOME:-$HOME/.codex}/config.toml" 

Claude's memory files are plain markdown that the docs say you can read, edit or delete. They are also excluded from Claude Code's transcript cleanup, so they stay until you or Claude removes them. OpenAI calls Codex's files generated state and advises against hand-editing them as your main control, so read them, then act through /memories or Delete Codex memories.

Reading once tells you what is there today. To see what changed between runs, put a copy under git. This script copies both stores into a private local repo and commits a snapshot. It removes the nested .git inside memories_v2/ after copying, so git records the memory files and not a pointer to another repo, and it copies the memory folders only, not Codex's sqlite state. It is a sketch: read it before you run it, and never push that repo anywhere, because memory files hold notes about you and your work.

#!/bin/sh # snapshot-agent-memory.sh # Copy agent memory into a private local git repo # so you can diff it between runs. Never push it. set -eu SNAP="$HOME/agent-memory-snapshots" CODEX="${CODEX_HOME:-$HOME/.codex}" mkdir -p "$SNAP" cd "$SNAP" [ -d .git ] || git init -q rm -rf claude codex mkdir claude codex for d in "$HOME"/.claude/projects/*/memory; do if [ -d "$d" ]; then cp -R "$d" "claude/$(basename "$(dirname "$d")")" fi done for d in memories memories_v2; do if [ -d "$CODEX/$d" ]; then cp -R "$CODEX/$d" "codex/$d" rm -rf "codex/$d/.git" fi done git add -A git commit -qm "snapshot $(date +%F-%H%M)" || true 

Run it before a day of agent work and again after, then read what the agents taught themselves:

cd ~/agent-memory-snapshots git diff HEAD~1 -- claude codex 

For a repo where you want every run to start from the same text, switch memory off there. Claude Code takes "autoMemoryEnabled": false in the project's .claude/settings.json, or CLAUDE_CODE_DISABLE_AUTO_MEMORY=1 in the environment. Codex documents memories.use_memories and memories.generate_memories:

[memories] use_memories = false generate_memories = false 

Memory is fine for you. Rules belong to the repo

Memory isn't useless. Claude's docs sell it as a way to "learn from your corrections without manual effort," and nobody wants to repeat the same preference every morning. How you like summaries written, which ticket you're on this week, where your dashboard lives: that is personal, and a machine-local note is a fine home for it.

If a fact changes what code ships, it is a rule, and rules go in a file the team can diff. The example in Claude's own docs is "always use pnpm, not npm," which Claude saves to auto memory when you ask it to remember. That is the kind of fact CI and your teammates need too. Say "add this to AGENTS.md" instead of "remember this," and the rule lands in a diff with your name on the commit.

Then make the snapshot diff a weekly habit. Anything in it that reads like a rule gets promoted into AGENTS.md through a PR and deleted from memory. Anything you don't recognise gets deleted, full stop.

Where Hermes fits

I build Hermes IDE, so weigh this section with that in mind. Hermes runs each agent's own interface in a real terminal. That means Claude Code's auto memory and Codex's memories behave in a Hermes session the way they do in your shell, and everything above still applies.

Hermes has a store of its own too. The README lists memory and context pins: "facts and pinned files saved for a project reach every later session's agent, also after a restart." They're saved per project, so they reach later sessions on that project. The rule from the last section applies to them as well: a fact every teammate needs goes in AGENTS.md, not in a pin on one laptop.

Where Hermes helps is the reviewable side. Every agent turn is saved, so you can diff or restore any single turn, with undo. If an agent edits the repo's AGENTS.md or .claude/settings.json halfway through a task, that edit sits in the turn that made it, where you can see it and roll it back. The Review Desk (⌘G, Ctrl+Shift+G on Windows and Linux) shows changes by turn or by file.

The Review Desk (⌘G) by turn: two turns for a Claude Code task, turn 1's diff of src/db.ts, the changed files and a commit message drafted from the turns.

Hermes is free, source-available under BSL 1.1, and runs on macOS, Windows and Linux. It isn't affiliated with Nous Research, Anthropic, OpenAI or any other agent vendor. It does not give you a diff of what Claude or Codex wrote into their own memory folders, or into config files in your home directory. For that, use the script above.

Open your agent's memory folder today. If you find a rule in there you never meant to give it, I'd like to hear what it was.

If this was useful

Memory, instruction files and guardrails are the parts of an agent that decide what it does when nobody is watching. The AI Agents Pocket Guide covers those patterns: how agents carry state between runs, and where to put the checks that hold no matter what the model remembers. It's one of 8 short books in the Pocket Guides for Developers series, on agents, RAG, prompts, observability, events, databases and system design: the whole series. If you run several agents on one repo, Hermes IDE is free to try, and it's on GitHub.