Backend
SHAPE: Discrete Supervision and Parallel Continuous Streams for Learning Dexterous Manipulation from Egocentric Human Data
arXiv cs.RO0 views
arXiv:2611.24585v1 Announce Type: new Abstract: Egocentric human recordings are far cheaper than dexterous robot demonstrations, promising stronger pretraining and direct skill learning without task-specific robot data. Yet humans and robots differ in morphology and kinematics, so human recordings lack the joint positions and actuator commands that supervise robot policies, providing only retargeted end-effector (EEF) actions. This raises two questions: how human data should supervise the policy, and how a dexterous robot should read out actions at deployment. We address both in SHAPE (Supervision from Human Action tokens with Parallel Executable streams), a vision-language-action (VLA) model whose backbone predicts discrete action tokens of EEF chunks as an auxiliary task, while a shared flow-matching expert outputs two executable streams, EEF and joint. On a bimanual robot with 22-DoF hands, a matched study shows that discrete supervision helps learn from human data, driving backbone gradients and shaping action-informative representations across domains. Under standard control pipelines at deployment, direct joint readout excels at fine-grained control with robot demonstrations, scoring 0.80 across six tasks against 0.63 for {\pi}0.5, whereas IK-based EEF readout excels at human-to-robot transfer. Pretrained with human data and discrete tokens, SHAPE learns new tasks from human demonstrations without task-specific robot data.
Source:https://arxiv.org/abs/2611.24585PreviousTinyARM-X: A Series of Open-Source, Configurable, and Transparent Torque-Controlled Desktop-Scale Robotic ManipulatorsNextHydrodynamics of robotic micro-swimmers in confinement
Related posts
HINT-WAM: Hindsight-Informed Training for World Action Models
0arXiv cs.RO
OmniFish: A Bioinspired, Modular, Reconfigurable Soft Robotic Fish Enabling Multi-Modal Swimming
0arXiv cs.RO
How to Calculate Your Instagram Engagement Rate
0dev.to AI
Building a Tech MVP in India: Real Lessons from Zero to Production Launch
0dev.to AI

Giving AI Agents a Memory That Lasts: A Practical Look at Amazon OpenSearch Memory
0dev.to AWS
The Invisible CloudFront Distribution: When AWS Amplify Manages CDN Infrastructure Behind the Scenes
0dev.to AWS