Backend
FFMT: Frequency-Factorized Multi-Rate Tokenization for Music-to-Dance Generation
arXiv cs.SD0 views
arXiv:2611.07384v1 Announce Type: new Abstract: Dance motion combines slowly varying body configurations with rapid local movements. We investigate how explicit temporal-frequency decomposition can guide the temporal resolution and codebook capacity of discrete motion representations. We introduce frequency-factorized motion tokenization (FFMT), which separates root-relative motion into a low-frequency choreographic scaffold and a complementary high-frequency residual. FFMT uses a lower token rate and a smaller codebook for the low-frequency component, and a higher token rate and a larger codebook for the high-frequency residual. This representation further induces a plan-then-articulate autoregressive factorization: low-frequency tokens establish coarse choreography, while high-frequency tokens realize local motion details conditioned on the generated plan and music. Experiments on AIST++ and FineDance show strong distributional fidelity while preserving realistic diversity and music--motion synchronization. Ablations consistently support frequency factorization, asymmetric temporal allocation, and low-to-high conditioning.
Source:https://arxiv.org/abs/2611.07384PreviousLatent Terrain: Adapting Neural Audio Autoencoders as Design Materials in NIMENextAdaptive and High-Fidelity Matching Algorithm for Retrieval-Based Accent Conversion
Related posts
Do verifiable metrics overestimate RSI?
0Hacker News New
Fixing a 25 dB LR1110 LNA gain bug on the Elecrow ThinkNode M9 (WadaMesh)
0Hacker News New
Show HN: Obeya, a spatial canvas for directing coding agents
0Hacker News Show
Show HN: Ascope – Daily App Store and Google Play keyword ranks, native Mac app
0Hacker News Show
Show HN: small WireGuard mesh VPN with no coordination server
0Hacker News Show
Show HN: Pots – Open-source Grok Bot alternative, used by 6 startups for a month
0Hacker News Show