arXiv：cs.AI· Yibo Wen, Han Liu·· 6 小时前
Autonomous Science Bench：评估 AI 智能体的自主科学发现能力
Autonomous Science Bench: Evaluating AI Agents for Autonomous Scientific Discovery
AI 导读
研究者推出 Autonomous Science Bench（v0.1），包含六个基于已发表实验的预算受限 design-build-test-learn 任务，覆盖生物学、化学与材料科学，智能体需选批次设计、接收实验室测量结果并交付预测模型。
正文
Abstract:AI agents for autonomous scientific discovery must decide which experiments to run under a limited budget. Most benchmarks for science agents provide fixed datasets, while others that support active experimentation largely rely on simulation. We introduce Autonomous Science Bench, an initial release comprising six budgeted design-build-test-learn campaigns grounded in published experiments across biology, chemistry, and materials science. Agents select batches of designs, receive measurements from a lab service, and use the results to guide subsequent experiments before delivering a predictive model. Success requires passing two calibrated gates: a discovery gate, which evaluates the designs the agent measured, and a learning gate, which evaluates its model's ranking of held-out designs. In a pilot evaluation of eight agent-model configurations and 144 trials, the best configuration succeeds in 67% of trials, while the hardest campaign is solved in only three of 24 attempts. The two gates reveal different weaknesses: some agents build accurate rankings but find too few promising designs, while others find promising designs but fail to predict which unseen designs will perform well. These differences are obscured by overall success rates alone. We release version 0.1 of the campaigns, lab runtime, and public leaderboard at this https URL.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18449 [cs.AI] |
| (or arXiv:2610.18449v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18449 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yibo Wen [view email]
[v1]
Tue, 13 Oct 2026 14:18:56 UTC (158 KB)
来源：arXiv：cs.AI · arxiv.org