arXiv：cs.AI· Yue Zhao, Zixiang Xu, Angela Huyue Zhang, S. Alex Yang, Xiyang Hu·· 16 小时前
agent-risk：面向 AI 智能体变更的风险评估框架
agent-risk: Risk Assessment for Changes to AI Agents
AI 导读
针对已部署 AI 智能体的变更审批，agent-risk 将经典风险分析扩展到部分日志记录和部分变更场景：重放已记录 episode，仅假设其余条件不变且智能体不预判变更。
正文
Abstract:Before changing a deployed AI agent (a stricter safety gate, a new model), organizations must approve the change, starting from existing logs. Extending classical risk analysis to partial records and partial changes to running systems, agent-risk replays logged episodes, assuming nothing else changes and the agent does not anticipate the change. The log settles episodes acting as before, a declared stop value settles stopped ones, and the rest need logged support, a saved-state rerun, or a bound. We prove what logs must record for exact replay and which saved states serve a gate family. For independent episodes, we bound the reruns a decision needs and tolerable drift from live behavior. In a bash sandbox, a 7,200-episode log alone decided neither of two retry gates, which let the agent act after refusals the log never shows. Rerunning only each gate's blocked episodes from saved states reached both live decisions at half the agent calls of a live test sized post hoc for them, given the log. Without per-step harm, a stopping gate's certified harm ceiling rose up to 2.5-fold, since the log hides whether harm preceded the stop. Approval thus needs evidence designed in advance: log fields, saved states, targeted reruns.
| Comments: | 43 pages, 5 figures, 7 tables. Code, registrations, and logs: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.16280 [cs.AI] |
| (or arXiv:2610.16280v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16280 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yue Zhao [view email]
[v1]
Mon, 12 Oct 2026 07:05:34 UTC (347 KB)
来源：arXiv：cs.AI · arxiv.org