Frontend
VOKE: Visual Object Kinematics Estimator for Dexterous In-Hand Manipulation
arXiv cs.RO0 views
arXiv:2611.16423v2 Announce Type: replace Abstract: Vision-based dexterous in-hand manipulation requires knowing not only where an object is, but also how it moves. However, most robotic manipulation systems rely on off-the-shelf visual perception models whose outputs are limited to static object masks, geometry, or 6D poses, leaving downstream controllers to infer object dynamics implicitly from noisy visual observations. We present VOKE, a kinematics-aware perception framework that augments existing object and hand-object pose estimators to recover a complete object kinematic state comprising pose, linear and angular velocity, and linear and angular acceleration. VOKE contains two core perception modules and a downstream application study. First, a Pluggable Kinematics Module (PKM) attaches to the output of a standard pose estimator and predicts the object's linear and angular velocity and acceleration from temporal visual features. Second, a Kinematics-guided Pose Refinement Module (KPRM) converts this motion vector into a second-order, motion-prior reference pose and refines the per-frame pose estimates. We evaluate VOKE on DexYCB and OakInk, where it consistently improves object pose estimation across three different backbones and substantially reduces velocity and acceleration estimation error compared with pose differencing. As a downstream application, we use Kinematics-Augmented Reinforcement Learning (KARL) to compare a kinematics-augmented policy with a pose-history baseline. After training the policies in simulation, we transfer the kinematics-augmented policy to the physical robot without policy fine-tuning, demonstrating the complete perception-to-control pathway in the target domain.
Source:https://arxiv.org/abs/2611.16423PreviousEmbodiExp: Compositional Experience Transfer for Embodied AgentsNextHEIR: Harness Egocentric Intent for Human-Robot Interactions
Related posts
From Meeting Transcript to Task Queue: Put a Verification Gate in the Data Contract
0dev.to AI
Réserver la place d’une image avant son chargement
0dev.to CSS
7 Arabic Website Localization Mistakes Developers Should Avoid
0dev.to CSS
100 tools in the browser, 9 on a 4 GB server: how we decide where a web tool runs
0dev.to WebDev
Building a face analysis tool that never uploads the photo
0dev.to WebDev
Pricing a multi-country trip from a public per-GB rate feed (and why the cheapest rate isn't your price)
0dev.to WebDev