AI & ML
Self-Supervised Object-Centric Representation for Multi-Task Audio Analysis
arXiv cs.SD0 views
arXiv:2610.41870v1 Announce Type: new Abstract: This paper studies the capability of an object-centric audio representation to support multiple downstream tasks within a unified framework. The object-centric model is trained in a self-supervised setup using slot attention by reconstructing Audio Spectrogram Transformer (AST) patch embeddings from a set of latent slots. It is trained on synthetic audio mixtures, which provides controlled multi-source audio scenes. These slot representations are evaluated for the classification, detection, and separation of sound events by training task specific output layers. On the classification task, the model achieves an F1-score of 0.8, comparable to clip-level AST embeddings which achieve 0.7. For sound event detection, it reaches a macro F1-score of 0.7, outperforming clip-level AST embeddings at 0.4 and matching AST patch embeddings at 0.7, which encode finer time-frequency information. In source separation, it achieves an SI-SDR improvement (SI-SDRi) of 18.4 dB. Furthermore, in a zero-shot source separation setting, the model achieves an SI-SDRi of 14.5 dB. These results demonstrate that slot-based representations capture transferable and compositional structure in audio scenes, enabling generalization across tasks and datasets.
Source:https://arxiv.org/abs/2610.41870PreviousMFlowAudio: Efficient Text-to-Audio Synthesis via Mamba-based Stateful Flow MatchingNextA General Theoretical Framework for Transformed Geometric Pattern Matching in Music
Related posts
Do AI coding agents care about SemVer?
0Microsoft DevBlogs

The Verge’s guide to Black Friday 2026
0The Verge

We’re giving away a bag filled with almost $900 worth of tech for Black Friday
0The Verge

Here are the shipping and return policies for all the big-name retailers
0The Verge

Here’s when the Black Friday sales start at Walmart, Target, and other retailers
0The Verge

How to shop like a pro during Black Friday and Cyber Monday
0The Verge