AI & ML
Probabilistic Relevance Decisions for Semantic Retrieval
arXiv cs.IR0 views
arXiv:2611.08014v1 Announce Type: new Abstract: Dense similarity supplies reusable corpus representations, while probabilistic relevance decisions offer query-dependent document scoring. We compare dense-only BGE-M3 with TypeSafe Jev Noul and OpenAI Decisions Predicate scores on the official BEIR SciFact test benchmark. The study evaluates 300 official test queries using identical positive-enriched 100-document pools and full-corpus dense retrieval followed by reranking at candidate depths of 50 and 100. Unlike the controlled pools, neither dense prefix receives positive injection. Jev has the highest observed Recall@5 and nDCG@10 under the evaluated conditions. At depth 50, full-corpus nDCG@10 is 0.6413 for BGE-M3, 0.7545 with OpenAI and 0.7805 with Jev. Extending depth from 50 to 100 increases known-positive candidate recall but produces small, uncertain final-effectiveness changes. All four paired-bootstrap depth intervals include zero. Unique-condition API scoring cost approximately doubles, while compatible cached decisions reduce incremental expenditure. OpenAI returns many exact-zero scores, and secondary exact-tie randomization reveals ranking sensitivity without replacing deterministic primary results. Top-100 latency mixes cached and new execution. Historical hybrid latency is a saved-component sum under different execution environments and timer boundaries. These exploratory, unadjusted analyses support relevance decisions as a complement to dense retrieval within this protocol, without establishing general superiority, probability calibration or equivalence between depths.
Source:https://arxiv.org/abs/2611.08014PreviousMEMOIR: A Training-Free Memory Lifecycle for Long-Horizon LLM AgentsNextDomain-specific information retrieval
Related posts

Two more Garmin smartwatches are down to record-low prices
09to5Google
Four Hidden Config Settings That Stop Token Leaks and Unapproved Writes in OpenAI Codex and Claude Code
0dev.to
Measuring Real Fulfillment Gains From Warehouse Sorting Automation Before You Scale Up
0dev.to
What to Check Before Committing to a Robotic Laser Welding Equipment Partner
0dev.to

How Akai and Moog are embracing vibe-coded mods
0The Verge

Big Tech lost big in the 2026 election
0The Verge