Open Source
TalkFlow: Structured Latent Flow Matching for Speech Driven Holistic Motion Generation
arXiv cs.GR0 views
arXiv:2610.40972v1 Announce Type: new Abstract: Holistic co-speech motion generation aims to synthesize coordinated facial expressions, hand gestures, and body movements from speech, which is challenging due to heterogeneous part-specific dynamics, long-range temporal coherence, and cross-part coordination. Directly modeling high-dimensional motion can be unstable, while overly compact or unstructured latent spaces may lose fine-grained motion details. We present \textbf{TalkFlow}, a structured latent flow framework for speech-driven holistic motion generation. TalkFlow learns a continuous three-stream latent representation for upper-body motion, lower-body motion, and facial expressions, preserving part-specific dynamics without forcing heterogeneous motion components into a single latent space. To maintain holistic coordination, lightweight cross-stream interactions are incorporated in both latent representation learning and flow-based generation. Conditioned on multi-level speech features, TalkFlow performs speech-conditioned latent flow matching, learning a continuous transport process from source latents to realistic motion latents. Experiments on the BEAT2 dataset demonstrate strong performance across facial expression accuracy, body gesture quality, and speech-motion synchronization. Qualitative comparisons and user studies further show that TalkFlow generates more natural, expressive, and speech-aligned holistic motion than state-of-the-art methods. The project page is available at https://xinmuwang.github.io/TalkFlow/.
Source:https://arxiv.org/abs/2610.40972PreviousBest 7 Real-Time AI Face Swap Tools in 2026: A Buyer's GuideNextDynamic Skinning: Kinematics-Driven Cartoon Effects for Articulated Characters
Related posts
Can't Beat Them... Join them?
0dev.to
How I Extracted Data from 783 PDFs with 97% Verified Accuracy (A Data Engineering Case Study)
0dev.to Python
Letagents – let your coding agents talk across machines
0Hacker News New
Show HN: Edgepad – Turn an Android phone into a trackpad for your laptop
0Hacker News Show
19 MB proves a 400B model reply at 72x cost
0dev.to Machine Learning
Show HN: Autorun-audit – find what runs when you open, CD into or install a repo
0Hacker News Show