AI & ML
Any Channel Count, One Token Stream: Incorporating Spatial Audio Understanding and Reasoning into Large Audio-Language Models
arXiv cs.SD0 views
arXiv:2611.18112v1 Announce Type: new Abstract: Audio phase relations encode source locations, yet existing large audio-language models (LALM) reduce audio to single-channel content to stabilise general audio understanding, leaving spatial information for direction, distance, and source-relation questions unused. We present SpheraAudio, which accepts recordings with any channel count -- mono, stereo, First-Order Ambisonics (FOA), or higher -- and maps them into a single token stream. A spatial encoder turns complex multichannel spectra into spatial tokens interleaved with audio tokens at corresponding temporal positions. Spatial Multimodal RoPE (SMRoPE) makes each token's modality identity and temporal slot visible to rotary attention without altering the language model's original RoPE. Training runs in three stages -- spatial-encoder pretraining, projector alignment, and joint supervised-and-reinforcement learning -- progressing from geometric supervision to language-based spatial decisions, with answers trained as explicit spatial chain-of-thought: step-by-step geometric reasoning over the heard scene. Current benchmarks tie questions to a fixed channel-count format and score answers without spatial reasoning evidence. We introduce SpheraQA, which spans mono, stereo, and FOA recordings and organises questions hierarchically from single-attribute perception through cross-source relations to multi-step reasoning, scored programmatically against simulated scene geometry and real-recording annotations with step-by-step solution references. Across SpheraQA and public spatial-audio benchmarks, SpheraAudio sets a new performance standard for spatial audio understanding. It supports high-precision localisation, tracking, and reasoning over static and moving sound sources while preserving strong general-audio capabilities. We will release code, SpheraQA, and evaluation metrics to support reproducibility.
Source:https://arxiv.org/abs/2611.18112PreviousCoTTAGE: An Explainable Judge for Text-to-Audio GenerationNexteDIAR: Training-Free Open-Speaker Streaming Diarization for Long Meetings and Large Archives via e-means
Related posts
Building an AI Agent Backend with FastAPI: From User Request to AI Response
0dev.to AI

SoduWork Custom Model Setup: Connect an OpenAI-Compatible Gateway and Handle the 180-Second Stall
0dev.to AI
AI startup hiring snapshot, 16 November 2026: 6,481 open roles, half at four companies
0dev.to AI
EC2 已经停止,账单为什么还在变?用资源清单代替猜测
0dev.to AWS

Amazon Black Friday 2026:5 Key Metrics and a Step-by-Step Data Collection Guide
0dev.to AWS
How to Trim Audio for an Online Course Lesson — Free, in Your Browser
0dev.to WebDev