HuggingFace Daily Papers·· 2 天前
LLM 智能体能否学会预判自身失败？基于结果置信度校准
Can an Agent Learn to Predict Its Own Failure? Outcome-Grounded Confidence Calibration in LLM Agents
AI 导读
研究提出在 GRPO 下用多步任务的实际结果校准 LLM 智能体每步的置信度信号。在 Qwen2.5-7B 的 ALFWorld、TextWorld 和 WebShop 离线 RL 中，步置信度对任务结果的 AUROC 从接近随机升至 0.68-0.85，置信度最高 10% 的步骤成功率约为平均的 4.3 倍。
正文
Abstract:Controlling an acting LLM agent means deciding, at each step of a long-horizon task, whether to trust its next action, abstain, or hand off to a human. That decision needs a per-step confidence signal predicting whether the current rollout will eventually succeed. The agent's own self-reported verbalized confidence is the natural candidate, but existing reinforcement learning methods calibrate it against the correctness of a decision at the current step. We propose a method to calibrate it against the eventual outcome of a multi-step episode: under GRPO, the group of rollouts for a task supplies success labels, and a proper scoring rule scores each step's confidence against the realized outcome. In offline RL on recorded trajectories of a Qwen2.5-7B agent in ALFWorld, TextWorld, and WebShop environments, the AUROC of step confidence against episode outcome rises from near chance to 0.68-0.85, and the top-10% of steps the agent is most confident about succeed ~4.3x more often than average. In online RL on the live ALFWorld agent, four calibration rewards improve early-step AUROC, expected calibration error goes from 0.42 to below 0.12, and selective execution improves over both the untrained agent and a format-only RL control, without degrading task success. We also introduce an evaluation protocol that separates episode depth, task difficulty, and attempt-level foresight, which shows where the gain lies: the trained agent recognizes from its first step which tasks it will fail, while telling a succeeding attempt from a failing one at the same task is harder, with some early-window signal emerging on TextWorld.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.18614 [cs.AI] |
| (or arXiv:2610.18614v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18614 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ashutosh Hathidara [view email]
[v1]
Tue, 13 Oct 2026 16:00:01 UTC (201 KB)
来源：HuggingFace Daily Papers · arxiv.org