AI & ML
WorldBridge: Bridging Video and Action for Task Generalization in World Action Models
arXiv cs.RO0 views
arXiv:2610.40748v1 Announce Type: new Abstract: Achieving the broad task generalization of embodied models in physical environments remains a central challenge in robot learning. We study this challenge in World Action Models (WAMs) through two complementary capabilities: video generalization, which predicts the visual evolution of new tasks, and action generalization, which translates these predictions into executable behavior through a video-to-action bridge. By varying video and action supervision for tasks outside an initial training set, we examine their respective contributions to task transfer. On RoboTwin 2.0, using only in-distribution video-action supervision yields a 25.07% success rate on out-of-distribution tasks. Introducing videos of these tasks without their action labels raises success to 55.90%, while adding just one action-labeled demonstration per task further improves it to 70.70%. These findings suggest that broader video coverage enables task transfer by leveraging existing inverse-dynamics capabilities, while even sparse action supervision further strengthens those capabilities. Guided by this perspective, we introduce WorldBridge, a WAM with separate video and action experts connected through structured attention and trained with joint flow matching. WorldBridge incorporates visual memory at two temporal scales and action forcing, alongside a unified data processing pipeline that standardizes and filters heterogeneous robot trajectories and egocentric hand-derived supervision. Across multiple manipulation benchmarks, WorldBridge achieves state-of-the-art performance among the compared methods, including 96.23% on RoboTwin 2.0 full data setting and 80.9% on clean to random setting, 82.8% on RoboCasa GR1 TableTop, and 57.6% on DOMINO Level-1. Our study highlights the complementary roles of video prediction and action grounding in developing robots that generalize across tasks.
Source:https://arxiv.org/abs/2610.40748Previous12 California Mobile App Development Agencies for Startup and Enterprise BuyersNextCaPtain: A Pipeline for Post-Training Code-as-Policy Robot Agents
Related posts
Merge builds product and business tools with OpenAI technology
0OpenAI Blog
How to Survive (Not Thrive) as a Salaryman
0HackerNoon
Debugging macOS LaunchAgent Failures With launchctl and Exit Code 78
0HackerNoon

Building Data Observability for Banking Pipelines
0HackerNoon
OpenMatter Network Positions Decentralized Computing as an Alternative To Data Centers for AI
0HackerNoon
BMW bumps battery for the better: The 2027 i7 60 xDrive, driven
0Ars Technica