arXiv：cs.AI· Seyedarmin Azizi, Erfan Baghaei Potraghloo, Arya Fayyazi, Saeid Shokoufa, Massoud Pedram·· 4 小时前
减少循环次数会消除循环深度语言模型的拒绝行为
Running Fewer Loops Removes Refusal Behaviour in Recurrent-Depth Language Models
AI 导读
循环深度语言模型在运行时减少循环次数可降低成本，但会大幅削弱安全性。Ouro-1.4B-Thinking 在训练深度一半时 GSM8K 准确率保留 89%，而对有害请求的协助率从 0.038 升至 0.205，增加 5.3 倍。线性分类器在单次循环后仍能以 0.967 的 ROC AUC 识别有害请求，说明缺失的循环去掉的是应随之而来的拒绝行为；在随机深度上训练可恢复安全性，且不损失全深度准确率。
正文
Abstract:A recurrent-depth language model reuses one block of weights several times in a row. The number of repetitions, which we call loops, is chosen when the model runs, not when it is trained, and running fewer is cheaper and faster, so a deployment may choose to do it. Running fewer loops also makes these models less safe, and the loss of safety is larger than the loss of anything else we measured. We study four released Ouro checkpoints and Huginn-0125. At half of its trained depth, Ouro-1.4B-Thinking still solves 0.692 of the GSM8K maths problems, against 0.775 at full depth, keeping 89% of its accuracy. Over the same reduction, the rate at which it helps with harmful requests rises from 0.038 to 0.205, 5.3 times higher. The information is not lost: a linear classifier reads whether a request is harmful from the model's internal state after a single loop, with an ROC AUC of 0.967 against 0.986 at full depth. What the missing loops remove is the refusal that should follow. We measure the effect with four instruments, one generating no text at all, and reproduce it on a second model family. Removing a single direction from Ouro-1.4B's internal state at full depth drops its refusal rate on harmful prompts from 0.950 to 0.130, identifying what the shallow model is missing. Two run-time repairs do not help: the models' early-exit rules do not spend more loops on harmful requests, and choosing a depth per request does not recover the loss. Changing the training does. Standard safety training makes the shallow model safe only by refusing 0.865 of harmless requests, while training at randomly chosen depths reaches a separation of +0.399 between harmful and harmless refusal at one loop, against +0.118. On Huginn-0125 it keeps harmful help at or below 0.040 at every depth from 1 to 32 loops, where standard training reaches 0.194, and it costs no accuracy at full depth.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.15218 [cs.AI] |
| (or arXiv:2610.15218v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15218 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Erfan Baghaei Potraghloo [view email]
[v1]
Sat, 10 Oct 2026 23:17:01 UTC (230 KB)
来源：arXiv：cs.AI · arxiv.org