AI & ML
21Codec: Decoder-side Speaker Control for Text-to-Speech
arXiv cs.SD0 views
arXiv:2611.24094v1 Announce Type: cross Abstract: We propose 21Codec, a streaming codec model with reduced recoverable speaker information from the token stream and decoder-side speaker control. Existing codec models infer speaker identity from token stream. Making them streamable is not sufficiently robust to speaker drift, because each chunk re-infers the identity from the local chunk context alone. Our adversarial speaker classifier and NANSY-style formant-shift data augmentation pushes the model to rely on the provided speaker embedding while reducing the recoverable speaker information from the token stream. We show that 21Codec can control speaker identity through the conditioning vector, and that the speaker identity is maintained across chunks during streaming. With the ability to control speaker identity even in mid-stream, 21Codec is a promising candidate for streaming TTS systems for better controllability of speaker identity.
Source:https://arxiv.org/abs/2611.24094PreviousBeyond Intelligibility: Fine-Grained Perceptual Assessment of Dysarthric Speech with Auditory Large Language ModelsNextLocation Matters: Layer-wise Prompt Conditioning for Voice--Style Disentanglement in Text-to-Speech
Related posts
Teaching VLMs What to Say, Not How to Reason: Rethinking Counterfactual Reasoning in Autonomous Driving
0arXiv cs.RO
SearchDiffusion: Information-Guided Diffusion Planning for Object Search
0arXiv cs.RO
Proprioceptive Simultaneous Base and Terrain Elevation Estimation for Legged Robot Locomotion
0arXiv cs.RO
Remote Teleoperation of Endovascular Robots: A Network Emulation Framework for Systematic Performance Evaluation
0arXiv cs.RO
QDD-MM: A Simple and Compact Open-Source Muscle Module with Quasi-Direct-Drive Motor for Tendon-driven Robots
0arXiv cs.RO
Old Experience, New Lessons: Relabeling Autonomous Robot Rollouts in Interactive Imitation Learning
0arXiv cs.RO