AI & ML
ProSearch: Benchmarking Multi-Constraint Protocol Retrieval in Experimental Science
arXiv cs.IR0 views
arXiv:2610.41396v1 Announce Type: new Abstract: In AI for Science scenarios, experimental design and method reproduction require retrieving protocols that satisfy multiple interdependent constraints, yet existing benchmarks lack systematic evaluation of this capability. To address this gap, we introduce ProSearch, a benchmark for multi-constraint protocol retrieval in experimental science. ProSearch has three key features: (1) realism, with queries grounded in real experimental records and expert-defined templates; (2) complexity, with 60 experimental fields and their dependencies; and (3) diagnosticity, with 3-5 explicit constraints per query, including numerical and negation constraints. We construct ProSearch through LLM-based extraction, data-driven constraint selection, controlled query generation, and evidence-based relevance validation. The resulting benchmark contains 1,515 queries across six experimental science domains. We conduct extensive experiments on ProSearch with 19 representative retrievers and complementary analyses. The best-performing retriever achieves an average nDCG@10 of only 0.290, which improves to 0.474 with reranking. Structured diagnostic analysis shows that explicit constraint matching provides a strong ranking signal, while manual error analysis identifies joint constraint binding as a key bottleneck. ProSearch provides a testbed for systematically evaluating scientific retrieval systems.
Source:https://arxiv.org/abs/2610.41396PreviousBrowseFlow: Sparse Event Flow Matching for Generative User SimulationNextHow BERT-based Cross-Encoders Model Relevance in Information Retrieval?
Related posts
Running AI Models Locally in 2026: Ollama, vLLM & Quantization
0dev.to

Most companies don't think they could leave their main cloud provider quickly should the worst happen
0TechRadar

'Nothing will ever replace the emotional depth and creative inspiration of human beings as artists and creators' — Call of Duty: Modern Warfare 4 composer speaks out against AI in video games, says 'm
0TechRadar

Two dead, five injured in Taiwan PCB factory explosion
0Tom's Hardware

HomeView will support third-party AI chatbots, suggests code
09to5Mac

Google’s AI search fumbles leave schools in the lurch
09to5Google