arXiv：cs.LG· Shicheng Liu, Adam Kahirov, Qi Zhang, Zhimin Hu, Song Wang, Junhong Lin, Julian Shun, Yada Zhu·· 3 小时前AI 评分41
VisAudit：评估多模态智能体的可视化诊断与修复能力
VisAudit: Evaluating Multimodal Agents for Visual Diagnosis and Repair
AI 导读
多模态智能体在自主审查可视化方面存在明显短板，为此研究者推出 VisAudit 基准，覆盖诊断修复、自主修复和开放世界验证三条赛道，包含 21 种图表类型、10 类缺陷的 1900 个缺陷实例及 300 张初始正确图表。实验显示，表现最好的多模态模型在自主修复设置下仅能完整修复 47.4% 的缺陷图表。
正文
Abstract:Multimodal agents are increasingly used for data visualization tasks but remain limited in autonomous review. Unlike humans, they may fail to recognize when a visualization is incorrect, determine what to change, repair it without disrupting correct content, and verify whether the intervention succeeded. Existing benchmarks largely evaluate predefined individual capabilities such as chart generation, instruction-guided editing, or defect detection, and therefore do not capture this gap in autonomous review. We introduce VisAudit, a benchmark for evaluating visualization diagnosis, repair, and verification. Given a rendered chart and configurable auxiliary evidence, including its source data table, intended text summary, and visualization code, an agent iteratively diagnoses potential defects, modifies and executes visualization code, inspects execution and visual feedback, and determines when no further intervention is needed. VisAudit defines three tracks spanning diagnosed repair, autonomous repair, and open-world verification, and contains 1,900 flawed instances across 21 chart types and 10 flaw categories, together with 300 initially correct charts. We construct the benchmark through controlled perturbations of validated source visualizations, with systematic verification and human-aligned quality control to ensure that injected defects are well-defined and recoverable from the available evidence. Experiments with leading multimodal models reveal a substantial gap from reliable autonomous review: the strongest evaluated model fully recovers only $47.4\%$ of flawed charts in the autonomous-repair setting.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.02399 [cs.LG] |
| (or arXiv:2610.02399v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.02399 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shicheng Liu [view email]
[v1]
Thu, 1 Oct 2026 19:26:35 UTC (249 KB)
来源：arXiv：cs.LG · arxiv.org