AI & ML
One from Infinity: Actualizing Futures from Pretrained World Models into Robot Actions
arXiv cs.RO0 views
arXiv:2609.36413v4 Announce Type: replace Abstract: A pretrained video world model admits many plausible futures for a scene, but a robot must realize the exact task-conditioned one. To turn world models into executable robot policies, existing methods fine-tune the heavy world model backbone using large-scale robot data and computational resources. Challenging this status quo, we argue that the expensive part has already been paid in the world model pretraining since the representation space of a video world model encoder lays out the diverse potential futures. In this case, what remains is to determine the future that accomplishes the task and to read out the actions that realize it. We formalize this task as actualization, which learns a task-conditioned realization on top of a prior supplied by a frozen world model encoder. This can be solved by a tiny actualizer model. We implement RoboActualizer with as few as 60M parameters on top of a frozen world model encoder. The actualizer is composed of two lightweight DiT experts that jointly predict future latents and actions by flow matching. The model can be trained entirely on a single GPU with 32 GB peak memory. With up to 100x fewer trainable parameters than existing WAMs and VLAs, RoboActualizer reaches great performance on simulation benchmarks including LIBERO, LIBERO-Plus, RoboTwin 2.0 and five tasks on two real-world platforms, with a low latency of 39 ms that allows real-time control.
Source:https://arxiv.org/abs/2609.36413PreviousComparative Evaluation of an XR Pen-based Control Interface for Semi-Autonomous Mobile Robot Navigation in Service EnvironmentsNextLinear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey
Related posts
What Is a Print-Ready PDF and How to Check Customer Files
0dev.to Tutorial
The Easiest Way to Update SaaS Video Docs Without Re-Recording Your Screen
0dev.to Tutorial
How to get a data analyst internship
0dev.to Career
Name the one thing you keep not finishing. Put a date on it.
0dev.to Career
PHP prevê atualizações críticas de segurança para 19 de novembro
0dev.to PHP
French tarot for 5 players, explained: calling a king and scoring a hand
0dev.to Beginners