AI & ML
Back to Mel: Learning to Bridge Fidelity and Perception in Generative Speech Enhancement
arXiv cs.SD0 views
arXiv:2611.22628v1 Announce Type: new Abstract: Recent generative techniques have facilitated the rapid proliferation of speech enhancement approaches built upon learnable compact latent representations and large generative backbones. Nevertheless, it remains inherently challenging to balance between acoustic fidelity, perceptual quality, and processing efficiency. In this paper, we revisit the Mel spectrum as a structured acoustic representation and propose BAME, a flow-matching-based framework that exploits the complementary priors for generative speech enhancement task. First, leveraging the explicit hierarchical frequency distribution of Mel spectrum, we elaborately devise a subband-aware Transformer (SaT) for efficient generative modeling. Using only 2.9% of the parameters, the proposed SaT achieves superior enhancement performance compared with a 331 M DiT backbone. Second, motivated by range-null decomposition theory of Mel filtering in the magnitude-spectral domain, we introduce a phase-aware neural vocoder that incorporates degraded-phase prior, together with a quality-guided dual-phase strategy to adaptively mitigate the effects of unreliable phase information. Third, we devise a simple yet effective one-step fidelity-perception alignment strategy that fine-tunes the flow model with complementary constraints from fidelity and perception domains. Extensive experiments are conducted on DNS-Challenge, EARS, GSR, and URGENT benchmarks, and both objective and subjective results demonstrate promising performance on various intrusive and non-intrusive evaluation metrics. These results further advocate the potential of exploiting structured acoustic representations for reliable and efficient generative speech enhancement. Audio samples and codes are available at https://github.com/AndongLi-speech/BAME.
Source:https://arxiv.org/abs/2611.22628PreviousBeyond Mimicry: Constrained Exploration with GRPO for Joint Multi-Talker ASR and Diarization under Unknown Speaker CountsNextSleep2Word: Peripheral-to-EEG Translation via Duration-Aware Wordization
Related posts
Wine Merges Remote OpenGL Swapchain Image Presentation, CEF Now Works With Wayland
0Phoronix

I lied to ChatGPT and Gemini and pretended to be an expert
0TechRadar

The overlooked technology that could determine the ROI of enterprise AI
0TechRadar

Microsoft is giving agents their own Cloud PC capabilities to help you with crucial tasks
0TechRadar

The Shure MV360 is an affordable mic for spatial audio capture
0TechRadar

Chief AI Officers are now a common sight in businesses — but new study finds they often don't really have any power
0TechRadar