AI & ML
Probabilistic Relevance Decisions for Semantic Retrieval
arXiv cs.IR0 views
arXiv:2611.08014v1 Announce Type: new Abstract: Dense similarity supplies reusable corpus representations, while probabilistic relevance decisions offer query-dependent document scoring. We compare dense-only BGE-M3 with TypeSafe Jev Noul and OpenAI Decisions Predicate scores on the official BEIR SciFact test benchmark. The study evaluates 300 official test queries using identical positive-enriched 100-document pools and full-corpus dense retrieval followed by reranking at candidate depths of 50 and 100. Unlike the controlled pools, neither dense prefix receives positive injection. Jev has the highest observed Recall@5 and nDCG@10 under the evaluated conditions. At depth 50, full-corpus nDCG@10 is 0.6413 for BGE-M3, 0.7545 with OpenAI and 0.7805 with Jev. Extending depth from 50 to 100 increases known-positive candidate recall but produces small, uncertain final-effectiveness changes. All four paired-bootstrap depth intervals include zero. Unique-condition API scoring cost approximately doubles, while compatible cached decisions reduce incremental expenditure. OpenAI returns many exact-zero scores, and secondary exact-tie randomization reveals ranking sensitivity without replacing deterministic primary results. Top-100 latency mixes cached and new execution. Historical hybrid latency is a saved-component sum under different execution environments and timer boundaries. These exploratory, unadjusted analyses support relevance decisions as a complement to dense retrieval within this protocol, without establishing general superiority, probability calibration or equivalence between depths.
Source:https://arxiv.org/abs/2611.08014PreviousMEMOIR: A Training-Free Memory Lifecycle for Long-Horizon LLM AgentsNextDomain-specific information retrieval
Related posts
🚀 The Adral Studio: Watch Us Architect a Complete OLX Clone Live
0dev.to Beginners

A Packing Manifest for Phu Quoc in March
0dev.to Beginners

Arduino Giga R1 WiFi MOSFET Module
0dev.to Beginners
Earn Money Testing Websites and Apps: The Complete TestLancer Guide
0dev.to Testing
I checked 29 student AI and dev-tool deals against the official pages. Here's what UK students can actually get
0dev.to OpenSource
ElevenLabs eleven_v3: 271 calls, $0.5716 in total, no failures
0dev.to Programming