AI & ML
ToolDex: An Ego-Exo Human Demonstration Dataset for Creative Physical Tool Use
arXiv cs.RO0 views
arXiv:2611.09036v1 Announce Type: new Abstract: Creative tool use is a fundamental human capability and an important step toward general-purpose embodied intelligence. It requires agents to reason about object affordances, physical constraints, and manipulation strategies, especially when standard tools are unavailable. However, progress in this area is limited by the lack of real-world datasets that capture human tool-use behavior with sufficient scale, fidelity, and diversity. Existing benchmarks are often simulation-based, focus on simplified interactions, or lack detailed observations and annotations. We introduce ToolDex, a large-scale egocentric-exocentric (ego-exo) human demonstration dataset for creative physical tool use. ToolDex captures scenarios in which humans select, adapt, and repurpose available objects as tools. Using synchronized egocentric cameras and eight exocentric cameras, the dataset provides high-fidelity multi-view observations and rich annotations for hands, tools, target objects, and their deformable interactions. Beyond data collection, we establish a benchmark suite characterizing three core capabilities: physical tool-use reasoning, sparse-view 3D and 4D reconstruction, and robot policy learning. ToolDex provides a foundation for studying human tool-use behavior and developing embodied agents that can adaptively use tools in complex real-world environments. Dataset visualizations and benchmarking results are available at https://tooluse-dataset.github.io/.
Source:https://arxiv.org/abs/2611.09036PreviousA Controlled Study of Models, Skills, Feedback, and Curriculum in Agentic RoboticsNextDobotTrack: Target-Grounded Vision-Language-Action Learning for Embodied Person Tracking
Related posts
Do Top-Ranked Local Businesses Get Recommended by ChatGPT? The November 9, 2026 Re-Check: 0 of 3
0dev.to AI
How to Buy ChatGPT Plus Cheap Without Getting Burned — Top-Ups, Pre-Made Accounts, and CDKs Explained
0dev.to AI

Implementing Capacity Providers in ECS on EC2 for Fully Automated Batch Processing Scaling
0dev.to AWS

Self-heal broken locators with a 2B model on your own GPU
0dev.to Python

AWS Config Costs Were Too High—I Cut Them in Half With One Configuration Change
0dev.to DevOps

Nominal, which builds software to make hardware manufacturing more efficient, raised $240M at a $3B valuation, up from $1B after raising $80M in March (Paayal Zaveri/Bloomberg)
0Techmeme