AI & ML
Self-Supervised Object-Centric Representation for Multi-Task Audio Analysis
arXiv cs.SD0 views
arXiv:2610.41870v1 Announce Type: new Abstract: This paper studies the capability of an object-centric audio representation to support multiple downstream tasks within a unified framework. The object-centric model is trained in a self-supervised setup using slot attention by reconstructing Audio Spectrogram Transformer (AST) patch embeddings from a set of latent slots. It is trained on synthetic audio mixtures, which provides controlled multi-source audio scenes. These slot representations are evaluated for the classification, detection, and separation of sound events by training task specific output layers. On the classification task, the model achieves an F1-score of 0.8, comparable to clip-level AST embeddings which achieve 0.7. For sound event detection, it reaches a macro F1-score of 0.7, outperforming clip-level AST embeddings at 0.4 and matching AST patch embeddings at 0.7, which encode finer time-frequency information. In source separation, it achieves an SI-SDR improvement (SI-SDRi) of 18.4 dB. Furthermore, in a zero-shot source separation setting, the model achieves an SI-SDRi of 14.5 dB. These results demonstrate that slot-based representations capture transferable and compositional structure in audio scenes, enabling generalization across tasks and datasets.
Source:https://arxiv.org/abs/2610.41870PreviousMFlowAudio: Efficient Text-to-Audio Synthesis via Mamba-based Stateful Flow MatchingNextA General Theoretical Framework for Transformed Geometric Pattern Matching in Music
Related posts
I built a daily GitHub AI leaderboard — 3 things nobody tells you about star counts
0dev.to GitHub
115-Million-Year-Old Blood Is a Data Persistence Story
0dev.to Data
WARN Act and Your 401(k): What Happens to Your Retirement Savings After a Layoff
0dev.to News
Overmoulding and Insert Moulding: When to Combine Metal and Plastic in One Part
0dev.to Design

Boost Mac Productivity: PopClip, Path Finder & Menuist
0dev.to Productivity
Probability-Conserving Flow Guidance
0arXiv eess.IV