arXiv：cs.CL· Josh Barua, Kyle Mahowald, William Rudman·· 16 小时前
视觉语言模型像有偏的贝叶斯观察者一样整合冲突证据
Vision-Language Models Integrate Conflicting Evidence Like Biased Bayesian Observers
AI 导读
研究从贝叶斯视角提出，视觉语言模型（VLM）会以模态特有的偏差，将来自各模态的条件独立证据与先验信念结合。所构建的两参数线索组合模型能准确预测 VLM 在未见过的输入退化组合下的偏好，并无需重新拟合即可迁移到新的不可靠形式。该框架将证据强度与整合偏差分离，解释了 VLM 为何既可能偏向文本、又可能给视觉证据更高权重。
正文
Abstract:Vision-language models must reconcile conflicting text and images. Prior work has characterized the resulting behavior in terms of modality biases--e.g., a "blind faith in text"--but why preferences vary across models, domains, and input conditions remains unclear. We develop a predictive account from a Bayesian perspective, positing that VLMs combine conditionally independent evidence from each modality with prior beliefs, subject to modality-specific biases. The resulting two-parameter cue combination model accurately predicts VLM preferences under unseen combinations of input degradation and transfers to new forms of unreliability without refitting. Separating evidence strength from integration bias explains how VLMs can follow text even while weighting visual evidence more heavily, reconciling reports of text and vision dominance under a single combination rule.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.19407 [cs.CL] |
| (or arXiv:2610.19407v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19407 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Josh Barua [view email]
[v1]
Wed, 14 Oct 2026 01:55:36 UTC (2,157 KB)
来源：arXiv：cs.CL · arxiv.org