arXiv：cs.AI· Lennart B\"urger, Piotr Komorowski, Moritz Weckbecker, Evgenii Kortukov, Aakriti Jain, Sebastian Lapuschkin, Wojciech Samek·· 17 小时前
研究：即使 CoT 是必要的，思维链监控也可能失效
Chain of Thought Monitoring Can Fail Even When CoT Is Necessary
AI 导读
前沿实验室用 CoT 监控检测 AI 系统的不当行为，但一项研究证明，即使 CoT 对答案而言是必要的，监控失败依然存在。作者提出四级 CoT 必要性层级，并在三个最低层级上通过干预实验测量 CoT 可监控性，在真实困难预测任务上均观察到监控失败；在困难的医学诊断任务中，患者性别和一项临床属性分别至少影响 19% 和 24% 的、带有必要 CoT 的正确回答，却几乎从未被明确表述。
正文
Abstract:Chain of Thought (CoT) monitoring is used by frontier labs to detect misbehaviour of AI systems. Yet, its reliability remains poorly understood. We attribute this to the lack of a successful organizing principle that leads to generalizable conclusions about monitoring reliability in different regimes. Recent work attributes monitoring failures to the CoT not being necessary for the answer, and argues that CoT becomes monitorable when it is necessary. We show that this binary necessary/not-necessary organizing principle is too coarse by demonstrating that monitoring failures persist even if the CoT is necessary for the answer. In response, we introduce a more fine-grained organizing principle based on a hierarchy of four CoT necessity levels. Building on this hierarchy, we develop a framework for measuring CoT monitorability at the three lowest levels via intervention experiments. We demonstrate monitoring failures at all three of these levels on realistic, hard prediction tasks. On a difficult medical-diagnosis task, patient sex and a clinical attribute influence the answer in at least 19% and 24% of correct responses with a necessary CoT, respectively, yet their influence is almost never articulated. These results show that even when CoT is necessary, it is not guaranteed to be monitorable.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.18105 [cs.AI] |
| (or arXiv:2610.18105v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18105 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lennart Bürger [view email]
[v1]
Tue, 13 Oct 2026 10:22:01 UTC (2,102 KB)
来源：arXiv：cs.AI · arxiv.org