AI & ML
FUSEMOS: Perceptual Evaluation of Text-to-Music Generation with Dual-Encoder Fusion and Ranking-Aware Composite Loss
arXiv eess.AS0 views
arXiv:2611.07562v1 Announce Type: new Abstract: Perceptual evaluation of generated music is essential for developing text-to-music (TTM) generation systems. Existing automatic music perceptual evaluation methods rely solely on the contrastive language audio pretraining (CLAP) model to extract both audio and text features for predicting human-annotated mean opinion scores (MOS), particularly for overall musical impression (OMI) and textual alignment (TA). However, CLAP is limited in capturing fine-grained musical attributes such as timbre and expressiveness \emph{etc}. To address this, we propose FUSEMOS, a dual-encoder fusion framework that integrates CLAP with MERT, a self-supervised model pretrained on large scale music audio, to better capture both high level semantic alignment and detailed musical characteristics for more accurate OMI and TA estimation. Furthermore, to overcome the limitations of conventional regression losses in measuring human preferred relative ranking of music segments, we design a novel ranking-aware composite loss function. Experiments conducted on the MusicEval benchmark demonstrate that our approach outperforms the existing SOTA methods in both mean squared error and rank correlation metrics by a large margin.
Source:https://arxiv.org/abs/2611.07562PreviousA Unified Multi-Encoder Framework for Music Lyric Intelligibility Assessment: Bridging Invasive and Non-Invasive ParadigmsNextAuditory and Visual Virtualization in 6DoF XR: Effects on Plausibility and Dynamic Auditory Distance Perception
Related posts
Show HN ::Duo Vibe–Couples App: Connect with yours and enjoy there
0Hacker News New
The Race to Build a "Find My" for AI Chips
0Hacker News New
Show HN: ISBN Books in Claude
0Hacker News New
OpenAI orginated from YC non-profit research? How to build impactful Institutes?
0Hacker News New
xAI's Jet Engine-Powered Data Center Has Experts Raising the Alarm
0Hacker News New
Aaronson-Ambainis Conjecture proved (By AI, of course)
0Hacker News New