AI & ML
Semantic Role-Structured World Model for Physical Interaction Planning
arXiv cs.RO0 views
arXiv:2608.22294v3 Announce Type: replace Abstract: World models for robotic manipulation must capture how actions transform a scene in ways that are meaningful for task execution. Existing object-centric approaches preserve the transient identity of entities, while leaving the evolving task structure that gives those dynamics meaning largely implicit. We introduce SemRole, a world model that represents scene evolution through semantic roles: a task-conditioned structure explicitly encoding the functional status of scene entities, their task-relevant relations, and the current phase of execution. Conditioned on language, visual observations, and proprioception, SemRole uses this structured state to govern action generation and action-conditioned dynamics, predicting interaction events, relation satisfaction, and phase progression. At inference time, these predictions guide phase-aware action selection and local repair of invalid rollouts. We instantiate SemRole on both lightweight LeWM and pretrained Cosmos backbones, improving closed-loop success from 45.5% to 83.0% on LIBERO, 42.9% to 55.2% on RoboCasa, and 55.0% to 81.0% on real-robot Galbot transfer. Across backbones and domains, explicitly modeling task-semantic structure yields substantially stronger world-model-based manipulation.
Source:https://arxiv.org/abs/2608.22294PreviousFlatness Preserves Instruction Following in Vision-Language-Action ModelsNextTies at the Top: A Point-in-Time Comparison of CVSS and EPSS at a Fixed Budget
Related posts

Your guide to containers at AWS re:Invent 2026
0AWS Containers Blog

Three Asian forwarders, one corridor: what Nippon Express, Yusen and CJ Logistics actually publish
0dev.to Data
Tương quan số liệu Nice vs PSG: Phép thử phòng thủ tại Allianz Riviera
0dev.to Data
Jacksonville, Florida Property Listings Dataset (2026-11-04): 6534 Rows, Excel Spreadsheet
0dev.to Data
Tương quan phong độ Parma vs AS Roma: Bài toán kiểm soát hỏa lực đội khách
0dev.to Data
Schalke 04 chạm trán Stuttgart: Bài toán hiệu suất chuyển hóa tại VELTINS-Arena
0dev.to Data