AI & ML
Self-Calibrating Vision-Agent-Action: Deploying Pick-and-Place to New Robot Cells without Demonstrations or Fine-Tuning
arXiv cs.RO0 views
arXiv:2611.19198v1 Announce Type: new Abstract: In simulation, Vision-Agent-Action (VAA) performs pick-and-place in robot cells with gripper or base-frame changes without robot demonstrations or fine-tuning. It uses pre-trained vision-language and language models for interpretation, measures the command map and camera geometry from its own motion and fingertips, estimates lens distortion, and plans grasps from measured object depth. A controlled study of eight public LIBERO-fine-tuned vision-language-action models (VLAs) varies robot hardware, camera geometry, appearance, and base frame. Gripper and frame changes dominate their losses in the tested fractional design. Across 18 cells with these changes, frozen VAA achieves 230/360 successes (64%), the refined VAA 268/360 (74%), and the strongest released VLA on this subset 76/360 (21%). Paired ablations support the self-calibration bundle, lens estimation under compounded changes, and the measured-depth pipeline. On a two-step LIBERO-Goal chain with object state carried between instructions, VAA completes 54/60 chains in changed cells, while four evaluated VLAs complete none.
Source:https://arxiv.org/abs/2611.19198PreviousEgo2Sim2Humanoid: Agentic Real-to-Sim for Learning Loco-Manipulation VLAs without Robot DemonstrationsNextContinuous Lower-Limb Joint-Angle Estimation from sEMG and IMU: The Conditional Value of Cross-Activity and Cross-Subject Data under Update-Matched Controls
Related posts
How OpenAI scales PostgreSQL to 800 million users
0dev.to Database
Spec-Driven PBT: Deriving Semantic Invariants from Code Specifications for Agentic Property-based Testing
0arXiv cs.SE
A Grey Literature Review of Graduate Students' Research Practices and Skills in the LLM Era
0arXiv cs.SE
ReBoT-MiCS: Reliability-Aware Bounding Time for Enhancing Timing Behavior of Embedded Mixed-Criticality Systems
0arXiv cs.SE
VoxelSBFL: Voxel-Level Spectrum-Based Failure Prediction for Vision-Language-Action Models
0arXiv cs.SE
Discontinuity-Aware Solvers for Vapor-Compression Cycles
0arXiv cs.NA