HuggingFace Daily Papers·· 2 天前
论文诊断 System One 模型在 Agent 循环中的决策失败：读对却做错
Reading Right, Acting Wrong: Diagnosing Decision Failures of System One Models in Agent Loops
AI 导读
一项新研究在覆盖模拟环境、客服策略、工具使用、网页导航与编程的冻结基准上，为每个下一动作问题搭配对同一状态的事实探针，诊断 System One 模型的决策失败。在十七个快速系统与一个推理模型上，多数错误动作发生在正确读取之后，且系统越准确这类错误占比越高。当探针事实可证明决定动作时，读对全部事实的模型仍无法组合它们，缺失的往往不是事实而是下一个子目标。
正文
Abstract:System One models choose an agent's next action by scoring a supplied set of options in a single pass. This makes them fast and cheap, but when they choose wrongly there is nothing to inspect: did the model misread what the agent has seen, or read it correctly and still act wrongly? We answer behaviorally. On a frozen benchmark of agent decision states from simulated environments, customer-service policies, tool use, web navigation and coding, we pair every next-action question with factual probes about the same state, each answered independently. Across seventeen fast systems and a reasoning model, the answer is consistent: most wrong actions follow correct reading, and the more accurate a system, the more of its errors take this form. These errors are not noise; they gather on the same states across systems of every kind. Where the probed facts provably determine the action, models that read every fact still fail to combine them, and their own answers, composed by code, repair the failure. On real states, what is missing is often not a fact but the next sub-goal.
| Comments: | Initial version released to encourage open discussion and share preliminary findings. Evaluations of additional open-source System One models are ongoing |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18579 [cs.AI] |
| (or arXiv:2610.18579v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18579 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yubo Li [view email]
[v1]
Tue, 13 Oct 2026 15:43:30 UTC (1,206 KB)
来源：HuggingFace Daily Papers · arxiv.org