AI & ML
REAL-2MIX: Evaluating Identity Faithfulness in Audio-Visual Target Speaker Extraction
arXiv eess.AS0 views
arXiv:2608.03964v3 Announce Type: replace Abstract: Audio-visual target speaker extraction should recover the person indicated by the video, rather than repeatedly return the acoustically dominant voice. We introduce REAL-2MIX, a Mandarin benchmark of jointly recorded two-speaker mixtures with synchronized multi-view video and scene-matched solo references. Its 77 scenes contain 7,598 clips; filtering yields 6,038 evaluable mixtures, each queried for both speakers, giving 12,076 paired target queries. We also curate VOXBLINK2-AVSE, comprising 250,828 synchronized audio--lip pairs. A seven-system TF-GridNet study examines visual representations, training corpora, and conditioning strategies. We evaluate transcription accuracy, speaker identity, residual overlap, and perceptual quality. The AVSR early-fusion system achieves 22.99% CER, and 93.38% of its outputs are more similar to the target speaker than to the interferer. Yet only 42.49% satisfy both this identity criterion and the overlap detector's single-speaker criterion; both target queries satisfy these criteria for just 20.77% of mixtures. A separate Real-AVSE challenge study examines development-remix adaptation. The benchmark reveals that favorable content and identity scores need not coincide with low detected overlap, motivating paired-target evaluation.
Source:https://arxiv.org/abs/2608.03964PreviousBridging Spiking and Continuous Dynamics in Spiking Neural Networks: A Muscle-Inspired ModelNextMeasuring Bounded Autonomy for an LLM Decomposition Planner on a Live 5G-Core Testbed
Related posts
The climate case for AI has not been made
0MIT Technology Review
I Let Pangram Loose on My Gmail Inbox. Here’s How Much AI It Found
0Wired
![Path Planning and Autonomous Navigation — Ch.24: Advanced Autonomous Behaviors [KR]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgmggiae4nyxpukfvgqba.png)
Path Planning and Autonomous Navigation — Ch.24: Advanced Autonomous Behaviors [KR]
0dev.to
화면이 사람을 붙잡는 힘 — AB 1709 소송이 개발자에게 던지는 질문
0dev.to
![Path Planning and Autonomous Navigation — Ch.12: Reversing and Parking [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F1k9m20qhwzb1csyeu5v1.png)
Path Planning and Autonomous Navigation — Ch.12: Reversing and Parking [EN]
0dev.to
![Path Planning and Autonomous Navigation — Ch.13: Docking and Charging [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ff66zllmrqlif2vsyczxb.png)
Path Planning and Autonomous Navigation — Ch.13: Docking and Charging [EN]
0dev.to