AI & ML
ReferClaw: Disambiguating User Intent for Egocentric Assistance
arXiv cs.MA0 views
arXiv:2611.07683v1 Announce Type: cross Abstract: Users of egocentric AI assistants often ask short, ambiguous questions whose intended referents cannot be uniquely resolved from video and dialogue history. For example, "Where did I leave it?" may refer to several previously handled objects. Existing VideoQA systems typically commit to a single interpretation, risking plausible but intention-misaligned answers. We introduce ReferClaw, an agentic framework that formulates egocentric streaming QA as an interactive process: the assistant asks clarifying questions to resolve user intent before answering. Three cooperating agents identify candidate interpretations, request clarification, and generate answers grounded in the resolved intent and visual evidence. A bounded, cost-aware policy balances expected information gain against user effort while limiting clarification turns and question complexity. Referclaw learns from interaction through intent-specific skill updates and process-reward-guided policy optimization with agent-level credit assignment. Evaluations on augmented egocentric QA benchmarks demonstrate improved answer accuracy and joint answer-and-grounding performance over existing baselines, alongside zero-shot transfer across datasets. These findings support interactive clarification as a practical approach to resolving online user intent in egocentric streaming assistance.
Source:https://arxiv.org/abs/2611.07683PreviousA Bilevel Neurodynamic Approach for Distributed SCAD-Regularized Sparse Least SquaresNextRadio-to-Pose Information for Gauge-Drift Correction in Cooperative Localization
Related posts
Partnering with Hacktron to find vulnerabilities in OpenAI’s systems
0OpenAI Blog
Free Chinese AI Models Are Getting Scary Good at Hacking
0Wired

Meta’s Muse AI can create custom ‘spaces’ for the version of the app you need right now
0The Verge
Meta introduces Spaces, a new way to collaborate with your Muse agent
0TechCrunch
GPU-Accelerated Weight Teleportation
0Hacker News
AI spend flatlined at +1.5% as Anthropic's $2T IPO shrinks
0dev.to Discuss