arXiv：cs.AI· Subhadeep Roy, Christoph Leiter, Steffen Eger·· 22 小时前
VisExMEM：基于视觉证据的可解释多模态评估指标
VisExMEM: Visual Evidence-Based Explainable Multimodal Evaluation Metric
AI 导读
研究者提出无参考评估器 VisExMEM，将图像描述拆解为原子断言、定位视觉证据并逐条验证，再平均为全局分数。基于 Qwen3.5 训练的 4B 和 9B 模型在八个基准上评测，VisExMEM-9B 的描述性综合分超过 VQAScore 11.7 分，并与 GPT-5.6 Sol 相当。同时发布 AGITA-Bench，含 600 个人工标注图文对和 8.7K 条原子断言。
正文
Abstract:Current image-text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Explainable metrics provide rationales, but the visual evidence supporting those rationales can still be difficult to verify. We introduce VisExMEM, a reference-free evaluator that decomposes captions into atomic claims, localizes visual evidence, verifies each claim, and averages claim-level support into a global score. Explanations use the same evidence and decisions. We train Qwen3.5-based 4B and 9B models on 100K samples with multi-task SFT, counterfactual refinement, and evidence-aware GDPO. Across eight benchmarks, VisExMEM-9B achieves the highest descriptive composite among open-source evaluators in our comparison, exceeding VQAScore by 11.7 points while remaining competitive with GPT-5.6 Sol. We also introduce AGITA-Bench, containing 600 human-annotated image-text pairs and 8.7K atomic claims for global and claim-level evaluation. For untuned Qwen3.5 and GPT-5.6 Sol controls, decomposed prompting raises the same composite by 3.6-5.1 points relative to holistic prompting. Human evaluation shows that VisExMEM explanations outperform prior explainable metrics and approach GPT-5.6 Sol.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.16441 [cs.CV] |
| (or arXiv:2610.16441v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16441 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Subhadeep Roy [view email]
[v1]
Mon, 12 Oct 2026 09:07:37 UTC (14,983 KB)
来源：arXiv：cs.AI · arxiv.org