arXiv：cs.AI· Aman Mehta, Anupam Datta·· 19 小时前
LLM Agent 过早定论问题：用隐藏状态诊断 Agent 的过早承诺
When Agents Commit Too Soon: Diagnosing Premature Commitment in LLM Agents
AI 导读
研究提出“表征承诺”指标，用重复运行中同一步隐藏状态的相似度衡量 LLM Agent 行为何时趋于稳定。在 Llama-3.1-70B 运行 ReAct 于 HotpotQA 上，第 4 步相似度越高，整条轨迹的变异性越低（r = -0.35），该模式在另外两个模型和 StrategyQA 上复现。
正文
Abstract:Tool-using LLM agents make early choices about what to search for, which evidence to trust, and what to do next. A wrong early choice can steer every later tool call, so an agent may produce a coherent but incorrect result. Existing evaluation does not reveal this process: final-answer scoring comes only after completion, and agreement across runs can be high even when every run is wrong. We ask whether hidden states reveal that repeated runs have already settled into stable behavior. Our contributions are threefold. First, we introduce representational commitment, which measures hidden-state similarity across repeated runs at the same step. On Llama-3.1-70B running ReAct on HotpotQA, we show that higher step-4 similarity is associated with lower whole-trajectory variability (r = -0.35); we find the same pattern in action sequences and replicate it on two other models and StrategyQA. This provides an internal measure for studying when agent behavior stabilizes. Second, we establish the measure's scope. Shared evidence explains much of the observed association, the HotpotQA result disappears on hard questions, and correct and confidently wrong questions overlap. We therefore show that the measure detects stability, not correctness, and cannot by itself establish that commitment was premature. Third, we evaluate monitoring and intervention. A monitor built from three partial runs predicts whole-trajectory variability (mean AUROC 0.81). A commitment prompt reduces variability by 28% relative to a token-matched filler, but not significantly relative to the standard agent after correction, and it does not improve accuracy. Together, these contributions provide a measurement and evaluation framework for studying how agent trajectories stabilize and for testing when additional evidence, sampling, or review may be useful.
| Comments: | 23 pages, 16 figures. Camera-ready version, Workshop on Agent Behavior at COLM 2026 |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2606.22936 [cs.AI] |
| (or arXiv:2606.22936v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.22936 arXiv-issued DOI via DataCite |
Submission history
From: Aman Mehta [view email]
[v1]
Mon, 22 Jun 2026 07:13:13 UTC (563 KB)
[v2]
Mon, 12 Oct 2026 19:16:13 UTC (382 KB)
来源：arXiv：cs.AI · arxiv.org