AI & ML
Learning Latent Goal Costs for Visual World-Model Planning with an Expanding Curriculum
arXiv cs.RO0 views
arXiv:2611.17529v1 Announce Type: new Abstract: Image-goal planning with a frozen visual world model depends on how predicted observations are scored against the goal. Mean-squared error (MSE) between embeddings from a pretrained visual encoder can provide a useful signal near a goal, yet favor states across an obstacle that are visually similar to the goal and difficult to reach. We fit a small residual adapter on top of frozen I-JEPA embeddings to improve this latent goal cost. Starting near each goal, a planner collects trajectories from progressively wider regions of initial states. The adapter learns from the order of observations along successful trajectories, with additional comparisons inferred from failed rollouts. No task-specific progress labels are used for fitting, so the procedure can be applied to other goal-reaching tasks. DotWall and PointMaze are used in this work. In paired 50-case evaluations with identical starts, goals, world models, and planner settings, success increases from 54% to 98% in DotWall and from 82% to 96% in PointMaze. On 16 unseen DotWall layout-goal cases, the adapter fitted on one fixed layout improves agreement between latent-cost ordering and shortest legal-path ordering from 71.9% to 85.8%. The results indicate that planning experience can improve a frozen visual representation's goal cost without retraining the encoder or the world model.
Source:https://arxiv.org/abs/2611.17529PreviousRacing the Clock: Teaching VLNs Budget-Conditioned Object SearchNextTalk to Act: A Synthetic Multi-Turn Dialogue Benchmark for Human Intent Extraction and Action Planning in Robotics
Related posts
Your Manager's Manager Has Booked a Skip Level Meeting With You
0dev.to

How to Make a Character Reference Sheet with AI (Prompts Included)
0dev.to Design
How Virtual Assistants Help Businesses Manage Their Work More Efficiently
0dev.to Productivity
DiffusionSSD: Achieving Extreme Visual Compression with Dual-stage Diffusion and Cross-level Near-Data Processing
0arXiv eess.IV
D2ToF: One-Step Diffusion for Depth Super-Resolution from Lightweight Single-Photon LiDAR Waveforms
0arXiv eess.IV
Few-Shot Open-Set Hyperspectral Pixel-Wise Classification with Spatial Aware Label Propagation
0arXiv eess.IV