AI & ML
Self-Supervised Object-Centric Representation for Multi-Task Audio Analysis
arXiv cs.SD0 views
arXiv:2610.41870v1 Announce Type: new Abstract: This paper studies the capability of an object-centric audio representation to support multiple downstream tasks within a unified framework. The object-centric model is trained in a self-supervised setup using slot attention by reconstructing Audio Spectrogram Transformer (AST) patch embeddings from a set of latent slots. It is trained on synthetic audio mixtures, which provides controlled multi-source audio scenes. These slot representations are evaluated for the classification, detection, and separation of sound events by training task specific output layers. On the classification task, the model achieves an F1-score of 0.8, comparable to clip-level AST embeddings which achieve 0.7. For sound event detection, it reaches a macro F1-score of 0.7, outperforming clip-level AST embeddings at 0.4 and matching AST patch embeddings at 0.7, which encode finer time-frequency information. In source separation, it achieves an SI-SDR improvement (SI-SDRi) of 18.4 dB. Furthermore, in a zero-shot source separation setting, the model achieves an SI-SDRi of 14.5 dB. These results demonstrate that slot-based representations capture transferable and compositional structure in audio scenes, enabling generalization across tasks and datasets.
Source:https://arxiv.org/abs/2610.41870PreviousMFlowAudio: Efficient Text-to-Audio Synthesis via Mamba-based Stateful Flow MatchingNextA General Theoretical Framework for Transformed Geometric Pattern Matching in Music
Related posts
Your RAG chatbot confidently answers questions about docs that don't exist in production. How to fix.
0dev.to

Chatbots might be coming to Apple’s HomeView
0The Verge
단백질 속으로의 여정: 아르키메데스 나사와 NexoPeek
0dev.to
Why Flattened PDF Tables Break RAG (and How to Preserve Cell Semantics in Markdown)
0dev.to

'Dubious Ownership': The 2022 Git CVE Behind the Error Your Agent's Container Hits
0dev.to Git
Git for Beginners: Stop Losing Your Code and Start Versioning Like a Pro
0dev.to Git