arXiv：cs.AI· Zhiyi Li, Sihan Hu, Tianning Xiao, Xiansheng Cai, Xiaojun Tan, Youjin Deng, Kun Chen·· 4 小时前
OpenProblemBench：在基础理论科学开放问题上评测 AI
OpenProblemBench: Benchmarking AI on Open Problems in the Foundational Theoretical Sciences
AI 导读
OpenProblemBench 是一个包含 82 道未解数学与理论物理问题的基准，每题附研究背景、假设与已有进展，由四个评估模型在无参考答案条件下独立判定解答的正确性、完整性与进展程度。在七个受测配置中，GPT-6-Astra 的平均判定解决率最高，为 14.0%，全尺寸开源模型为 5.5-6.7%，Flash 模型为 2.4-3.7%。
正文
Abstract:The next frontier for artificial general intelligence is tackling unresolved scientific problems, calling for benchmarks that assess progress beyond established knowledge. We introduce OpenProblemBench, a benchmark of 82 unresolved problems drawn from the mathematics and theoretical physics literature. Each problem supplies the research context, assumptions, and prior progress needed to investigate the question. We select problems whose proposed solutions admit comparatively clear checks of their decisive mathematical or computational claims. Four evaluator models independently assess the correctness, completeness, and degree of progress of each submission without reference solutions. Across seven evaluated configurations, GPT-6-Astra achieves the highest mean judged solve rate of 14.0%, compared with 5.5-6.7% for the evaluated full-size open models and 2.4-3.7% for Flash models. Case comparisons connect stronger outcomes to changes in problem representation, general arguments that extend beyond finite evidence, and proofs of the steps needed to complete a solution. By grounding evaluation in questions arising from the research literature, OpenProblemBench provides a setting for investigating the capabilities and limitations of AI as a contributor to foundational theoretical science.
| Comments: | 21 pages, 4 figures, including appendices |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.11118 [cs.AI] |
| (or arXiv:2610.11118v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11118 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sihan Hu [view email]
[v1]
Thu, 8 Oct 2026 02:43:51 UTC (80 KB)
来源：arXiv：cs.AI · arxiv.org