HuggingFace Daily Papers·· 2 天前
AdaExploreBench：评估 LLM 智能体在动态环境中主动探索与测试时适应能力的基准
AdaExploreBench: Benchmarking LLM Agents for Proactive Exploration and Test-time Adaptation in Evolving Environments
AI 导读
研究者提出 AdaExploreBench，用隐藏规则实例化环境动态，要求 LLM 智能体主动收集证据并推断规则，并在规则无预告漂移时从性能下降中察觉旧知识失效、重新探索。在八个前沿 LLM 上测试显示，GPT-6 Sol 仅达 61.9% 校准准确率；提供充足证据或移除规则漂移可使平均表现提升近 10%。结果表明当前智能体仍难以主动获取正确证据并识别其时效性。
正文 · 原文
Authors:Rui Wang, Zishen Lai, Jiayu Liu, Cheng Qian, Jeonghwan Kim, Heng Wang, Xuejun Zhang, Jiateng Liu, Bingxuan Li, Shujin Wu, Qing Zong, Tianshi Zheng, Chunyang Li, Qihan Lin, Chen Li, Dadi Guo, Xiusi Chen, Yangqiu Song
Abstract:Large Language Model (LLM) agents are increasingly deployed in unfamiliar and evolving environments, requiring them to proactively explore, understand, and adapt to environmental dynamics. However, existing benchmarks rarely test whether agents can proactively discover how an environment works and adapt as it evolves. To address this gap, we introduce AdaExploreBench, a benchmark for proactive exploration and test-time adaptation in evolving environments. By instantiating environmental dynamics as hidden rules, AdaExploreBench requires agents to actively acquire evidence and infer the rules. Under implicit rule drift, the latent rule changes without announcement, requiring agents to detect from degraded performance that prior knowledge is stale and re-explore. Experiments on eight leading LLMs show that proactive and adaptive exploration remain challenging: GPT-6 Sol achieves only 61.9% calibrated accuracy, while providing sufficient evidence or removing rule drift improves average performance significantly by close to 10%. These results reveal a critical bottleneck in current LLM agents: they can often reason effectively from informative evidence once it is available, but struggle to actively acquire the right evidence and recognize when previously acquired evidence is no longer valid.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.25950 [cs.CL] |
| (or arXiv:2610.25950v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.25950 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rui Wang [view email]
[v1]
Mon, 19 Oct 2026 07:41:50 UTC (559 KB)
来源：HuggingFace Daily Papers · arxiv.org