arXiv：cs.CL· Yulin Sun, Kele Xu, Yong Dou·· 3 小时前
ICAP-Gate：为大音频语言模型选择性控制音频影响
Selective Listening: Mechanism-Guided Control of Audio Influence in Large Audio-Language Models
AI 导读
ICAP-Gate 通过机制引导的任务条件控制，作用于各模型对干预敏感的late audio pathways，在 4 个 LALM、2 个推理基准、环境音与自然语音干扰下共 16 项全量评估中，Influence Rate 与 Answer Flip 均低于无门控推理。
正文
Abstract:Large audio-language models (LALMs) exploit multimodal evidence, yet task-irrelevant audio can alter text-reasoning decisions when listening is unnecessary. Aggregate Accuracy can hide this paired drift because audio-induced repairs and damages may cancel. Paired drift analysis and targeted interventions identify architecture-specific, intervention-sensitive late audio pathways as actionable control points. We introduce ICAP-Gate, which applies mechanism-guided, task-conditioned control to each model's pathway. Across four LALMs, two reasoning benchmarks, and environmental-sound and natural-speech interference, ICAP-Gate has lower point estimates for Influence Rate and Answer Flip than ungated inference in all 16 full-split model--condition evaluations. Fixed suppression degrades automatic speech recognition (ASR) across all four models, whereas ICAP-Gate matches ungated ASR performance by preserving the pathway for explicit audio-demand instructions. ICAP-Gate has lower paired-drift point estimates than mitigation prompting in all four evaluated settings and provides competitive stabilization relative to eight-sample Self-Consistency while using one generation per query; in controlled ARC measurements, Self-Consistency incurs $7.0$--$9.2\times$ ungated latency. These results establish selective modality influence control as a design principle for robust multimodal reasoning.
| Comments: | 24 pages, 5 figures, 20 tables |
| Subjects: | Sound (cs.SD); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.11196 [cs.SD] |
| (or arXiv:2610.11196v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11196 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yulin Sun [view email]
[v1]
Thu, 8 Oct 2026 04:00:47 UTC (2,253 KB)
来源：arXiv：cs.CL · arxiv.org