AI & ML
Flatness Preserves Instruction Following in Vision-Language-Action Models
arXiv cs.RO0 views
arXiv:2606.23641v2 Announce Type: replace Abstract: Vision-language-action (VLA) models have the potential for open-world generalization by leveraging pretrained vision-language representations, yet downstream finetuning on limited robot data often degrades these representations. This leads to brittle action policies that ignore language instructions in favor of visual shortcuts, despite the base VLM having this knowledge, which is a failure mode we term instruction blindness. We hypothesize that standard finetuning with limited data applies gradients to a sparse set of points, which manifests as a sharp loss landscape with high-curvature minima. We analyze this effect, then show that regularization through explicit flatness-preserving optimization is a simple yet effective technique while finetuning on the exact same data. Specifically, we demonstrate that applying sharpness-aware minimization (SAM) during VLA finetuning significantly improves instruction following by over 55% across multiple simulation and real-world benchmarks without additional data, architectural modification, or pretraining. Explicitly preserving flatness better preserves pretrained knowledge compared to other regularization techniques that simply anchor model weights or constrain learning. We further analyze the effect of selective sharpness, quantify its effects, and show that such approaches are complementary to existing guidance techniques.
Source:https://arxiv.org/abs/2606.23641PreviousOrchestrating Wheeled Mobile Robots Online without ConflictsNextSemantic Role-Structured World Model for Physical Interaction Planning
Related posts

Your guide to containers at AWS re:Invent 2026
0AWS Containers Blog

Three Asian forwarders, one corridor: what Nippon Express, Yusen and CJ Logistics actually publish
0dev.to Data
Tương quan số liệu Nice vs PSG: Phép thử phòng thủ tại Allianz Riviera
0dev.to Data
Jacksonville, Florida Property Listings Dataset (2026-11-04): 6534 Rows, Excel Spreadsheet
0dev.to Data
Tương quan phong độ Parma vs AS Roma: Bài toán kiểm soát hỏa lực đội khách
0dev.to Data
Schalke 04 chạm trán Stuttgart: Bài toán hiệu suất chuyển hóa tại VELTINS-Arena
0dev.to Data