Backend
MCGI: Manifold-Consistent Graph Indexing for Billion-Scale Disk-Resident Vector Search
arXiv cs.IR0 views
arXiv:2601.01930v4 Announce Type: replace Abstract: Graph-based Approximate Nearest Neighbor (ANN) search often suffers from performance degradation in high-dimensional spaces due to the Euclidean-Geodesic mismatch, where greedy routing diverges from the underlying data manifold. To address this challenge, this paper presents Manifold-Consistent Graph Indexing (MCGI), a geometry-aware and disk-resident indexing method that leverages Local Intrinsic Dimensionality (LID) to allocate the search effort of a graph index according to the intrinsic geometry of data. Specifically, MCGI estimates the LID around each query from the distances that its own search has already computed and resizes the candidate list of the query once, so that queries in high-LID regions receive longer lists while queries in low-LID regions terminate earlier; the estimate itself costs no additional disk reads. Theoretical analysis shows that the success probability of any fixed search list decays exponentially with the LID of the query. At matched high recall, MCGI raises the query throughput of DiskANN on the same standard Vamana index by up to $1.43\times$ on three million-scale benchmarks and by up to $1.24\times$ on the billion-scale SIFT1B and Deep1B.
Source:https://arxiv.org/abs/2601.01930PreviousENGRAM: From Agent Experience to Weights through Verified ConsolidationNextBenchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering
Related posts

System Design Interviews From Production Scars, Not Blog Diagrams
0dev.to Career
Pennsylvania Layoffs November 2026: WARN Numbers Explained
0dev.to Career

PHP Dies After Every Request. How the Zend Engine Actually Runs Your Code.
0dev.to PHP
No Sign-Up, No Accounts: The Privacy Posture of a Simple Downloader
0dev.to PHP
Compute crypto perpetual funding cost from a public API (Python)
0dev.to Beginners
What to work out before you send Pokémon cards to be graded
0dev.to Beginners