arXiv：cs.AI· Hongzhan Lin, Shidong Cao, Ziyang Luo, Wenhao Chai, Mong-Li Lee, Wynne Hsu·· 5 小时前AI 评分55
SafeActBench：从证据到行动，工具使用智能体如何失败
From Evidence to Action: How Tool-Using Agents Fail
AI 导读
研究者发布 SafeActBench，包含 656 个案例、六个操作领域和五种协议，用于追踪工具使用智能体从证据建立到行动执行的失败链条。在十个模型-框架配置上的分析显示，静态行动评估强并不代表交互式执行可靠，失败常始于执行之前，如调查不完整或在证据建立前就行动；单行动执行在证据齐备后通常可靠，多行动工作流还会暴露未解决的前置条件和执行不完整问题。
正文
Abstract:Tool-using agents make consequential changes to external state, yet correct outcomes do not guarantee that their actions were supported by evidence established beforehand. We study where this evidence-to-action chain breaks as agents move from deciding whether to act to executing single actions and dependent workflows. Across ten model-harness configurations, strong static action assessment can coexist with much weaker interactive execution. Failures often begin before execution: agents stop with incomplete investigation or act before required evidence is established. Once required evidence is obtained, single-action execution is usually reliable, while multi-action workflows additionally expose unresolved prerequisites and incomplete execution. For this analysis, we introduce SafeActBench, comprising 656 cases across six operational domains and five protocols that progress from static action judgment and investigated non-action to single- and multi-action workflows. A provenance-bound Evidence Ledger and deterministic trajectory evaluator track what information was established, when actions occurred, and whether downstream dependencies were satisfied. These results show that failures arise not only from missing information, but also from how agents use established evidence when deciding and executing actions.
| Comments: | 36 pages. Project page: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.07753 [cs.CL] |
| (or arXiv:2610.07753v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.07753 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lin Hongzhan [view email]
[v1]
Tue, 6 Oct 2026 04:50:29 UTC (3,665 KB)
来源：arXiv：cs.AI · arxiv.org