arXiv：cs.AI· Aotian Chen, Frazier N. Baker, Reza Averly, Shan Zhang, Tianao Chen, Daniel Adu-Ampratwum, Xia Ning, Yuan Xue·· 4 小时前
ChemExplain：面向解释性化学推理的多模态基准
ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoning
AI 导读
研究者发布 ChemExplain，一个从 383 篇同行评审论文构建的 1180 道化学多选题基准，每题配分子或反应图像、观察到的化学效应与四个候选解释，考查模型的“解释性归因”能力。最强模型准确率为 78.6%；在 600 道专家复核子集上，71.4% 的错误被归类为归因失败，这也是开源模型和化学专用模型最大的错误类型。
正文
Abstract:Current chemistry benchmarks primarily evaluate outcome prediction or reaction pathway generation, but rarely test whether a model can explain why an observed chemical effect occurs. We introduce ChemExplain, a benchmark of 1,180 multiple-choice chemistry questions drawn from 383 peer-reviewed papers. Each question pairs a molecular or reaction image with an observed chemical effect and four candidate explanations, each associated with a different structural feature visible in the image. Answering requires the model to perceive the depicted structure, apply chemical knowledge, and identify which feature most directly explains the effect, a capability we term explanatory attribution. The correct feature-mechanism relation is derived from supporting evidence in the source paper rather than invented by the question-generation LLM. We construct the benchmark automatically with a multi-agent framework that applies progressive quality control at three stages and validate it with eight chemistry domain experts. The strongest evaluated model reaches 78.6% accuracy, leaving considerable room for improvement. On a 600-question expert-reviewed subset, reasoning-trace analysis classifies 71.4% of the strongest model's errors as attribution failures. Attribution is also the largest error category for the open-source and chemistry-specific models included in our analysis.
| Subjects: | Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.17183 [cs.AI] |
| (or arXiv:2610.17183v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17183 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuan Xue [view email]
[v1]
Mon, 12 Oct 2026 17:53:56 UTC (4,960 KB)
来源：arXiv：cs.AI · arxiv.org