AI & ML
Robust Multimodal Learning via Entropy-Gated Contrastive Fusion
arXiv stat.ML0 views
arXiv:2505.15417v2 Announce Type: replace Abstract: Real-world multimodal systems routinely face missing-input scenarios, and in reality, robots lose audio in a factory or a clinical record omits lab tests at inference time. Standard fusion layers either preserve robustness or calibration but never both. We introduce Adaptive Entropy-Gated Contrastive Fusion (AECF), a single light-weight layer that (i) adapts its entropy coefficient per instance, (ii) enforces monotone calibration across all modality subsets, and (iii) drives a curriculum mask directly from training-time entropy. On AV-MNIST and MS-COCO, AECF improves masked-input mAP by +18 pp at a 50% drop rate while reducing ECE by up to 200%, yet adds 1% run-time. All back-bones remain frozen, making AECF an easy drop-in layer for robust, calibrated multimodal inference.
Source:https://arxiv.org/abs/2505.15417PreviousLinear Bandits on Ellipsoids: Minimax Optimal AlgorithmsNextOptimal In-context Adaptivity and Distributional Robustness of Transformers
Related posts
Is semantic chunking actually better than rule-based splitting for RAG retrieval?
0dev.to
Firefox RSS Reader: Get News Headlines on Your New Tab Without a Subscription
0dev.to

A slow WordPress admin: capture one request before changing plugins
0dev.to
Proton CEO dismisses AI security-test request as 'bullshit and loser thinking'
0Hacker News New
Musk and Gwynne Shotwell on AI Risks, Peer Review, Starship, Terafab, Merger
0Hacker News New
Anthropic sends out, then retracts details of IPO filing, emails show
0Hacker News New