Mem0 publishes 92.5% on LoCoMo. A third-party rerun of the same benchmark got 61%.

I spent two weeks reading every number this category publishes, and the most useful thing I found wasn't a faster memory layer. It was that the numbers everyone quotes describe a product you can't actually self-host.

The gap, with sources

Mem0's own research page reports 92.5% on LoCoMo and 94.4% on LongMemEval, at ~6,900 tokens per query. That's the number in every "best memory layer" list.

An independent paper (arXiv 2606.29778, Mandol) reruns LoCoMo with the published protocol and gets 61.00% (GPT-4o-mini) and 64.20% (GPT-4.1-mini) for Mem0. A 28–31 point gap.

That's not me calling anyone a liar. Mem0 says it plainly in its own docs: the published scores describe the managed platform, which "includes proprietary optimisations the SDK does not have." If you self-host the Apache-2.0 build, you are not running the thing that scored 92.5.

This matters because the reason most people look at self-hosting is data residency — and the self-hosted path is the one with the unmeasured number.

The same paper measured latency, which almost nobody quotes

Table 5 of the same paper, on an H800 at 10 QPS:

System Search mean P90 P99
Mem0 1,089 ms 1,397 ms 4,637 ms
Zep 571.7 ms 614.8 ms 5,348.7 ms
MemOS 440.5 ms 528.4 ms 777.1 ms
Mandol (their own) 82.2 ms 88.5 ms 94.8 ms

A 4.6-second P99 is what "memory" costs when it's an HTTP hop into someone else's vector store. For an agent that recalls on every turn, that's the whole interaction budget.

What a self-hosted memory layer actually costs you to install

Mem0 self-hosted: FastAPI + PostgreSQL with pgvector + Neo4j, three services kept in sync. Mimir: one ~8MB Rust binary with SQLite embedded, 36–44 MCP tools, MIT. Memir: local SQLite, zero-token writes, pip install. SuperLocalMemory: MIT, zero cloud. DeepMem: self-hostable Mem0-compatible API, paid tiers start at $1.9/month.

The floor in this category is free, and it's been free for a while. Anyone selling a memory layer on price is selling against $0.

So here's what I built, and what I can and can't claim

Cortex is a single 35.7MB Go binary with an embedded store, MCP-native, no external dependencies. My own reproducible numbers, on a Windows laptop, 557 markdown files / 3.1MB:

  • index build: 9.19 s
  • database size: 11.8 MB
  • end-to-end query (hybrid vector + BM25): 45–54 ms
  • rg on the same corpus for comparison: 43.4 ms

What I can claim: at 45–54 ms end-to-end, a query costs roughly 1/20th of Mem0's measured 1,089 ms mean and 1/86th of its P99. Same "server-side retrieval" shape, different order of magnitude.

What I cannot claim: that Cortex is more accurate than Mem0. I have not run LoCoMo or LongMemEval. I have no Recall@5 or MRR number on a public benchmark. If someone tells you their memory layer is "better" without a benchmark score, that's a marketing line, including mine.

That's the gap I'm working on next, and I'd rather say it out loud than let a commenter find it.

The one thing nobody in this category covers

Mimir's BM25 is porter-stemmed — English. Mem0's extraction is LLM-driven on English conversation. Neither segments Chinese. If your corpus is Chinese docs, meeting notes, or mixed CN/EN research material, you're feeding a tokenizer that doesn't know where your words end.

Cortex ships Chinese segmentation in the binary. That's not a benchmark claim, it's a capability claim, and it's the reason I stopped trying to win the "best memory layer" contest and started aiming at a smaller pond.

Reproduce it yourself

# Windows cortex install # index a folder cortex start # stdio MCP server 

The benchmark record, corpus size, and hardware are in the repo under bench/. If your numbers differ, I want the diff — that's more useful to me than a star.

If you want to test the claim

14-day trial, no card: https://cortex.qataroasis.cc/trial — it gives you a Pro license that expires, and the binary is the same one in the release. If it doesn't install, that's my bug, and I'll fix it in the thread.

Sources: mem0.ai/blog/ai-memory-benchmarks-in-2026 (self-reported scores); arXiv 2606.29778 Table 5 (independent latency + LoCoMo rerun); docs.mem0.ai/platform/pricing (managed-only scores, discontinued tiers); cognee.ai/mem0-alternatives (self-hosted stack composition); github.com/MakerViking/mimir (Mimir specs); local bench record at bench/2026-11-19.