arXiv：cs.AI· Bingyu Shen, Jiaqi Tang, Lan Wei, Yi Sheng, J. Jenny Li, Boyang Li·· 4 小时前
可执行的 Prefix Value：定位 LLM 智能体失败原因的新方法
Executable Prefix Values for Locating Agent Failures
AI 导读
研究提出用可执行方式区分 LLM 智能体失败轨迹中的"原因"与"症状"：从失败轨迹的中间状态重启智能体，估计每步的 prefix value V(h_t)，得到带认证的失败不可返回点（PNR）。
正文
Abstract:Work on debugging LLM agents treats the "first mistake" marked by a human or LLM judge as ground truth for where a trajectory failed. We ask whether such labels point at causes or symptoms, and make the distinction executable: we restart the agent from reconstructed intermediate states of a failed trajectory and estimate its prefix value V(h_t), the probability of success from each step, with a certified point of no return (PNR) for a failure. On 150 SWE-bench Verified tasks with two frozen open-weight agents, we build such curves for 47 and 60 failed trajectories and probe every step labeled by four frontier judges under the Traverse rubric. We find that the labels do not mark what a controller needs. Judges of different lineages agree on the exact step in only 21/57 cases, and the newest judges flag mistakes in 8/10 and 10/10 successful runs. For 66% of the labels, the agent's success probability does not drop at the labeled step; only 7/188 answers name the decisive step. Restarting at the measured restart point raises expected success from 0.44 to 0.57 offline, and real restarts confirm the advantage, whereas restarting at a judge's label does not (-0.08 to -0.03). We release our code, curves, and labels.
| Comments: | 46 pages, 12 figures, 60 tables |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.17633 [cs.AI] |
| (or arXiv:2610.17633v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17633 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bingyu Shen [view email]
[v1]
Tue, 13 Oct 2026 03:00:35 UTC (661 KB)
来源：arXiv：cs.AI · arxiv.org