Every vector database comparison starts with recall at a fixed latency, and every one of them concludes that the graph index wins. That conclusion is correct and almost useless, because it compares algorithms at a memory budget nobody constrained. In production the constraint is not latency, it is the RAM line on the invoice, and once you fix that, the ranking changes.
The clean way to think about it: an approximate index is a compression scheme with a quality dial, and the dial is priced in bytes per vector. HNSW keeps a graph in memory. IVF-PQ keeps compressed codes. Both are paying for recall, they just pay in different currencies.
Part One: The arithmetic of a billion vectors
A billion vectors at 768 dimensions in FP32 is 3.07 terabytes. Nobody serves that from RAM. So every production system compresses, and the compression ratio is the design decision.
At 8 bits per dimension you get 768 bytes per vector, 768 GB for the billion. At product quantisation with 96 subquantisers of 8 bits each, you get 96 bytes per vector, 96 GB. That is the difference between a machine you can rent and a machine you cannot.
HNSW on top of raw or lightly quantised vectors needs the graph too: at M=32, roughly 128 bytes per vector of links, plus the vectors themselves. IVF-PQ needs the codes plus a coarse quantiser, and the coarse quantiser is small. So the memory ratio between the two, at comparable quality targets, is roughly four to eight times.
Four to eight times is not a tuning difference. It is a different deployment shape.
Part Two: Where the curves cross
The plot above is my model at 95% recall@10 for one billion vectors, 768 dimensions. HNSW reaches 95% at around 2-3 GB per million vectors of resident memory, because the graph needs to be traversed exactly and the distance computations happen against full-precision vectors. IVF-PQ reaches the same recall with about 100-150 MB per million, but only when the codebook is well trained on a representative sample.
The crossover is not about recall, it is about the cost of the rerank. IVF-PQ returns candidates by approximate distance; to hit high recall you rerank the top 100 with full-precision vectors, which means you need those full vectors somewhere. If they live on SSD, you pay in latency. If they live in RAM, you have just given back most of the memory you saved.
This is the part that vendor benchmarks omit. IVF-PQ is cheap until you insist on the recall number, and then you buy the vectors back.
Part Three: The bill at matched recall
At 95% recall@10 on a billion vectors, my model puts HNSW at roughly 18 dollars per million queries across four memory-optimised nodes, IVF-PQ at 9 dollars across two, IVF-PQ with rerank at 13 across three, and a disk-based graph index at 11 across two.
The spread is smaller than the memory ratio suggests, for a simple reason: at fixed recall, the cheaper index does more work per query. IVF-PQ probes multiple inverted lists and computes many approximate distances. HNSW walks a graph and stops early. The compute cost partially cancels the memory saving.
So the honest conclusion is: the index choice moves the bill by 2x at most, and the workload moves it by 10x. A team that tunes the index while ignoring query distribution is optimising the small term.
Part Four: What actually decides
Three factors dominate, in this order.
Filtering. If 80% of your queries carry a metadata filter, the index that handles filters natively wins regardless of recall curves, because post-filtering destroys recall and pre-filtering destroys the graph structure. Most "HNSW is faster" benchmarks use unfiltered queries, which is not what production looks like.
Update rate. HNSW inserts are cheap and deletes are expensive; a graph with many tombstones degrades and needs periodic rebuild. IVF-PQ retraining the codebook is a batch operation you cannot run continuously. If your corpus changes hourly, the maintenance cost exceeds the query cost.
Freshness tolerance. If a document indexed five minutes ago must be findable, you need a small exact index in front of the big approximate one. That hybrid is what most mature systems end up with, and it is not in any benchmark.
Part Five: A deployment shape that holds up
Start with the cheapest index that meets recall at your actual filter rate, not at the benchmark's. For most teams that is IVF-PQ with a rerank over a hot subset.
Keep the full vectors in a columnar store on local NVMe, not in RAM. Rerank 50-100 candidates per query; that is a few hundred kilobytes of reads, which NVMe handles without noticing.
Put a small HNSW index over the last day's writes, sized to fit comfortably, and merge it nightly. This gives you freshness without rebuilding the main index.
Measure recall against your own queries weekly. Recall degrades silently as the corpus drifts away from the codebook's training distribution, and the only signal is a support ticket about a search that used to work.
The transferable lesson
Approximate nearest neighbour search is a compression problem wearing a data-structure costume. Every technique is trading bytes for accuracy, and the trade only looks favourable while the corpus is the shape the codebook was trained on.
The teams that get this right treat the index as a budget line with a quality target, instrument recall continuously, and accept that the answer changes as the workload changes. The wrong question is which index is faster. The right question is how many bytes per vector you are willing to pay for the recall your users actually notice, and that number is almost always smaller than the benchmark assumes.
SOL: HCqPmPJM3NVtS6YrmVaTZYnuP8XZ2PFgQZKT94VK1LkS
Creating this takes real effort — thank you for the support.
