AI & ML
FlowTouch: View-Invariant Visuo-Tactile Prediction
arXiv cs.RO0 views
arXiv:2603.08255v2 Announce Type: replace Abstract: Tactile sensation is essential for contact-rich manipulation tasks. It provides direct feedback on object geometry, surface properties, and interaction forces, enhancing perception and enabling fine-grained control. An inherent limitation of tactile sensors is that readings are available only when an object is touched. This precludes their use during planning and the initial execution phase of a task. Predicting tactile information from visual information can bridge this gap. A common approach is to learn a direct mapping from camera images to the output of vision-based tactile sensors. However, the resulting model will depend strongly on the specific setup and on how well the camera can capture the area where an object is touched. In this work, we introduce FlowTouch, a novel model for view-invariant visuo-tactile prediction. Our key idea is to use an object's local 3D mesh to encode rich information for predicting tactile patterns while abstracting away from scene-dependent details. FlowTouch integrates scene reconstruction and Flow Matching-based models for image generation. Our results show that FlowTouch is able to bridge the sim-to-real gap and generalize to new sensor instances. We further show that the resulting tactile images can be used for downstream grasp stability prediction. Our code, datasets and videos are available at https://flowtouch.github.io/
Source:https://arxiv.org/abs/2603.08255Previous$\pi$-StepNFT: Wider Space Needs Finer Steps in Online RL for Flow-based VLAsNextSpecification-Aware Distribution Shaping for Robotics Foundation Models
Related posts
Stopping Is a Capability: What Two Governance Talks at Google DevFest Atlanta 2026 Had in Common
0dev.to
Criei uma plataforma para desenvolvedores brasileiros. Agora vem a parte mais difícil: fazer ela crescer.
0dev.to
Agentic backdoors via pretraining poisoning
0Hacker News
Anthropic reveals $900B AI spending commitments ahead of IPO
0Hacker News
What is AI?
0dev.to AI
Cara Mengelola Grup Telegram dengan Bot AI Gratis: Solusi Lengkap untuk Pengembang & Pemilik Grup
0dev.to AI