AI & ML
PILOT: Guiding Long-Horizon World-Model Planning for Robot Manipulation with Imagined Subgoals and Policy Priors
arXiv cs.RO0 views
arXiv:2610.14335v2 Announce Type: replace Abstract: Latent world-model planning has shown promising results in goal-conditioned robotic manipulation, but remains challenging for long-horizon planning due to limited temporal guidance and inefficient action search. A distant goal image offers little guidance for preparatory actions, capturing intermediate subgoal images by hand is impractical, and uninformed sampling makes action search inefficient. We propose PILOT (Policy-Initialized Local Optimization with Temporal subgoals), which brings two foundation models into latent world-model planning: a video model turns a language instruction into visual subgoals, and a vision-language-action policy initializes the cross-entropy method (CEM). CEM refines action sequences toward each subgoal with an action-conditioned world model, and no goal image is needed at test time. With both models fine-tuned on 40 demonstrations, PILOT achieves 68.8% success on four real UR5e tasks, compared with 55.0% without policy initialization, 13.8% for policy-only execution, and 0.0% for direct final-goal planning. In simulation, PILOT exceeds direct execution of the same policy by 30 percentage points on long Push-T tasks and by 22 percentage points on LIBERO-10. Subgoals and the policy prior are complementary: better subgoals and stronger priors each raise success, and even a weak policy that succeeds alone on only 3.3% of medium Push-T tasks is an effective prior.
Source:https://arxiv.org/abs/2610.14335PreviousVideo2SwimFish: An Automated Pipeline for Reconstructing Controllable Fish Models and Biological Locomotion from Real Fish VideosNextRollout then Recycle: 3,718 Labelled Real-Robot Episodes from Benchmarking 7 Policies on 12 Tasks
Related posts
Show HN: Fast deterministic natural language search for OpenStreetMap data
0Hacker News
Sharp Tech on Video: Jensen’s Not Coming Back, AI Shopping, The iPhone Duo
0Stratechery

When a sync window closes: How Argo CD self-heal skipped the migration hook
0CNCF Blog
The "Unified Format" Nightmare in Enterprise IT (Air-Gapped Rooms & Double Timesheets)
0dev.to Watercooler
Phân tích thống kê Inter Miami vs Columbus Crew: Bài toán hàng thủ trước chuỗi phong độ thăng hoa
0dev.to Data
Biến Động Hàng Thủ: Barcelona Quan Tâm Lisandro Martinez, Man Utd Nhắm Nhân Sự Từ Fulham
0dev.to Data