AI & ML
Evaluating the Evaluation of Music Generation Models with Generative Psychometrics
arXiv cs.SD0 views
arXiv:2611.21809v1 Announce Type: new Abstract: Subjective user studies are one of the major methods for evaluating music generation models. However, subjective evaluation protocols vary widely and are rarely specified in enough detail to be reproduced, hindering comparison across studies. In this paper, we map subjective evaluation methods in ISMIR music generation papers from 2020-2025 and apply generative psychometrics to assess the stability of their items. We extract 284 items from the papers' methodological descriptions, using an LLM and manual validation to standardize them. We represent these items as contextual embeddings and apply Unique Variable Analysis to remove redundant, locally dependent items and Bootstrap Exploratory Graph Analysis to estimate their latent dimensions. Results show the network converges onto a taxonomy of six stable quality dimensions: Rhythm, Melody, Audio, Composition, Structure, and Lyrics, pointing to a unified methodology for conducting user studies to evaluate music generation models.
Source:https://arxiv.org/abs/2611.21809Related posts
Toward Planning Representations That Separate Task Structure from Visual Content
0arXiv cs.RO
Spectral Steering: Local Inverse Geometry for Frozen Diffusion Generators
0arXiv cs.RO
Representing Differential Inclusions as Monads: A Framework for Compositional Controller Synthesis
0arXiv cs.RO
AcousticBEV: Acoustic BEV Object Localization for Intelligent Vehicles
0arXiv cs.RO
Reliable Conditioned Hamilton-Jacobi Reachability via Subsolution Learning on Exact Local Windows
0arXiv cs.RO
Dependable Voice Command for Multi-UAV Swarms Using Small Language Models
0arXiv cs.RO