AI & ML
Self-Calibrating Vision-Agent-Action: Deploying Pick-and-Place to New Robot Cells without Demonstrations or Fine-Tuning
arXiv cs.RO0 views
arXiv:2611.19198v1 Announce Type: new Abstract: In simulation, Vision-Agent-Action (VAA) performs pick-and-place in robot cells with gripper or base-frame changes without robot demonstrations or fine-tuning. It uses pre-trained vision-language and language models for interpretation, measures the command map and camera geometry from its own motion and fingertips, estimates lens distortion, and plans grasps from measured object depth. A controlled study of eight public LIBERO-fine-tuned vision-language-action models (VLAs) varies robot hardware, camera geometry, appearance, and base frame. Gripper and frame changes dominate their losses in the tested fractional design. Across 18 cells with these changes, frozen VAA achieves 230/360 successes (64%), the refined VAA 268/360 (74%), and the strongest released VLA on this subset 76/360 (21%). Paired ablations support the self-calibration bundle, lens estimation under compounded changes, and the measured-depth pipeline. On a two-step LIBERO-Goal chain with object state carried between instructions, VAA completes 54/60 chains in changed cells, while four evaluated VLAs complete none.
Source:https://arxiv.org/abs/2611.19198PreviousEgo2Sim2Humanoid: Agentic Real-to-Sim for Learning Loco-Manipulation VLAs without Robot DemonstrationsNextContinuous Lower-Limb Joint-Angle Estimation from sEMG and IMU: The Conditional Value of Cross-Activity and Cross-Subject Data under Update-Matched Controls
Related posts

Sources: Nvidia made two separate introductions in 2022 connecting Alibaba with alleged Chinese spy Alice Huang, actively helping Megaspeed land Alibaba (The Information)
0Techmeme
Show HN: I cut an LLM's screen-watching cost to ~138 tokens per step
0Hacker News New
Codeberg's Domain Was Suspended
0Hacker News New
Gemini AI agents repeatedly bypassed safeguards to attack real website
0Hacker News New
A new AI agent builder. No-code visual builder with customer support platform
0Hacker News New
Jacob Tsimerman on AI can supercharge mathematics [video]
0Hacker News New