Industry
Seeing What Matters: Benchmarking Task-Conditioned Actionable Spatial Belief for Manipulation
arXiv cs.RO0 views
arXiv:2611.09794v1 Announce Type: new Abstract: When should a robot look at the cup, and when at the portafilter? Spatial-reasoning benchmarks evaluate whether a model can describe a scene, but manipulation needs something more specific: the ability to acquire the particular evidence that makes the current command executable. A view that is informative for one task can be useless for another, even at the same workstation. We introduce Seeing What Matters, a benchmark for task-conditioned actionable spatial belief in simulated tabletop manipulation. Across three domains--coffee workstation, drawer/cabinet, and microwave/appliance--the benchmark provides 860 frozen evaluation cases; expanding across methods, budgets, and inference repeats yields more than 115,000 evaluator episodes under two protocols: closed-loop actionability and same-object functional out-of-distribution generalization. Each case pairs a set of candidate RGB views with a manipulation command, shared scripted skills, and hidden postconditions, so the benchmark directly measures whether a view-selection policy acquires the observations its command actually needs. We also provide a diagnostic reference pipeline that maintains a spatial-functional belief state and ranks views by missing evidence. At primary budget settings, this pipeline reaches 98.8% closed-loop and 98.9% functional-OOD success; ablating the functional or command-binding components collapses performance to near zero. Seven vision-language models, evaluated on the same cases, reach only 26-43%. Replacing the reference pipeline's coverage metadata with frozen RGB-derived scores preserves 98.5%/98.9% success; shuffling the visual scores collapses it to 15%/11%, confirming that the result depends on image content. These gaps suggest that spatial belief for manipulation should be evaluated not by scene-level completeness, but by whether it acquires the evidence the current command requires.
Source:https://arxiv.org/abs/2611.09794PreviousA wearable input/output interface learns personalized models of haptic perceptionNextGrasp-TAGS: Task-Aware Grasp Selection via Critic Distillation for Prehensile Loco-Manipulation
Related posts
L'IA accélère la découverte de peptides antimicrobiens : et si nous apprenions enfin à voir les protéines en 3D ?
0dev.to Machine Learning

Inside Apple's Siri AI Lab: Schemas, Tools, and Skills
0dev.to Machine Learning
Removing ads, telemetry and AI extras from Windows 11: a practical checklist
0dev.to Tutorial
What IT certification exams actually cost in 2026: exam, retake and renewal
0dev.to Career
Beta Testing an App Before Launch on Google Play and the App Store: A 3-Week Plan
0dev.to Testing
Add ARKit blendshapes to MetaHumans via CLI tool
0Hacker News New