Frontend
VOKE: Visual Object Kinematics Estimator for Dexterous In-Hand Manipulation
arXiv cs.RO0 views
arXiv:2611.16423v2 Announce Type: replace Abstract: Vision-based dexterous in-hand manipulation requires knowing not only where an object is, but also how it moves. However, most robotic manipulation systems rely on off-the-shelf visual perception models whose outputs are limited to static object masks, geometry, or 6D poses, leaving downstream controllers to infer object dynamics implicitly from noisy visual observations. We present VOKE, a kinematics-aware perception framework that augments existing object and hand-object pose estimators to recover a complete object kinematic state comprising pose, linear and angular velocity, and linear and angular acceleration. VOKE contains two core perception modules and a downstream application study. First, a Pluggable Kinematics Module (PKM) attaches to the output of a standard pose estimator and predicts the object's linear and angular velocity and acceleration from temporal visual features. Second, a Kinematics-guided Pose Refinement Module (KPRM) converts this motion vector into a second-order, motion-prior reference pose and refines the per-frame pose estimates. We evaluate VOKE on DexYCB and OakInk, where it consistently improves object pose estimation across three different backbones and substantially reduces velocity and acceleration estimation error compared with pose differencing. As a downstream application, we use Kinematics-Augmented Reinforcement Learning (KARL) to compare a kinematics-augmented policy with a pose-history baseline. After training the policies in simulation, we transfer the kinematics-augmented policy to the physical robot without policy fine-tuning, demonstrating the complete perception-to-control pathway in the target domain.
Source:https://arxiv.org/abs/2611.16423PreviousEmbodiExp: Compositional Experience Transfer for Embodied AgentsNextHEIR: Harness Egocentric Intent for Human-Robot Interactions
Related posts

Planning Against the Model
0dev.to TypeScript
react-native-ultrastore v3.1.0: Zero-Dependency MMKV v4 State with On-Device Vector & Stream Cache
0dev.to TypeScript
Developer Seeks Real-World Use Cases and Framework Adoption Evidence for Standardized TC39 JavaScript Decorators
0dev.to JavaScript
React Hooks Explained: useState, useEffect, useRef & useNavigate
0dev.to JavaScript
Blurry canvas export: give preview and export separate render paths
0dev.to JavaScript

My JavaScript Learning Journey
0dev.to JavaScript