arXiv：cs.AI· Daniel J. Evans, J. P. Naiman, JooYoung Seo, J. Stephen Downie·· 16 小时前
多面板科学图表视觉问答数据集发布，用于评估 LMM 图表分析能力
Do you "see" what I "see"? A Multi-panel Visual Question Answer Dataset for Large Language Model Chart Analysis
AI 导读
研究者构建了一个面向大语言与多模态模型（LLM/LMM）科学图表分析的多面板视觉问答（VQA）数据集，任务按四级描述模型从基础概念理解延伸至解释与分析。测试发现，随着面板数量、绘图元素数量和问题难度增加，LMM 准确率下降；分析还引入成本相对性能的指标，并公开了问答对、随机绘图生成代码及图表元素边界框元数据。
正文
Abstract:Digital libraries increasingly host scientific publications whose figures encode primary research findings, yet making this visual content searchable, accessible, and reusable remains an open challenge. Large language and multimodal models (LLMs/LMMs) provide the possibility to automate many figure parsing tasks such as determining data trends or extracting data from plots. In this work, we consider if these models can reliably support task-oriented visual data analysis when presented with synthetic, but realistic, scientific figures that contain multiple charts. Our Visual Question Answering (VQA) process scaffolds our tasks along a four-level description model from basic conceptual answers to interpretive and analytical understanding. Additionally, we outline our process for synthetically generating our single and multi-panel (charts in which there are multiple panels present) plots and testing the data with various LMMs using quantitative analysis. In general, we find LMMs become less accurate with increasing figure complexity as the number of panels, number of plot elements, and difficulty of questions increase. Finally, because LMM accuracy is expected to change rapidly, our analysis also considers cost relative to performance as a metric to inform future user decisions. We provide the detailed question-answer pairs used in this analysis, along with the code used to generate random plots, question-answer pairs and detailed bounding-box metadata for chart elements. The code and data are available at this https URL and this https URL\_qa\_multipanel
| Comments: | 16 pages, 4 figures, accepted to The 30th International Conference on Theory and Practice of Digital Libraries, 2026 |
| Subjects: | Digital Libraries (cs.DL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2610.14173 [cs.DL] |
| (or arXiv:2610.14173v1 [cs.DL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14173 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jill Naiman [view email]
[v1]
Fri, 9 Oct 2026 18:00:02 UTC (272 KB)
来源：arXiv：cs.AI · arxiv.org