arXiv：cs.AI· Max Hartman, Vidhata Jayaraman, Moulik Choraria, Lav R. Varshney·· 20 小时前
推理模型的反蒸馏新方法 TARS：让防御难以被察觉
Hiding in Plain Sight: Detectability-Aware Antidistillation of Reasoning Models
AI 导读
研究人员提出反蒸馏方法 TARS（Thought Anchor RemovalS），通过移除推理轨迹中语义冗余的 thought anchors 来阻碍下游蒸馏，同时保持轨迹可读性。
正文
Abstract:Distillation via reasoning traces exposes frontier models to adversarial third parties, who can bypass their guardrails and misappropriate their capabilities. Existing antidistillation defenses perturb the teacher's sampling distribution to hinder downstream student learning, but these perturbations also degrade the trace quality even when task accuracy is preserved, often leaving visible artifacts such as syntactic incoherencies and repetition chains. Beyond eroding user trust, such traces are statistically distinguishable, allowing adversaries to filter or resample them and weaken the defense. An effective defense must therefore limit how visibly it degrades the teacher. To address this gap, we formulate antidistillation as a max-min game whose constraint set explicitly encodes defense detectability. Satisfying both axes--- degrading the student while remaining undetectable---may require more clever perturbations, akin to sparse but targeted adversarial attacks in robustness literature. We apply the same analogy to reasoning steps, targeting thought anchors: sentences identified in prior interpretability work as strongly influencing a model's output. We propose Thought Anchor RemovalS (TARS), which identifies and removes thought anchors, whose content is semantically redundant, so that the reasoning trace remains legible to the user. Modeling these removals as burst-deletions, we then provide formal guarantees on its detectability. Finally, we empirically demonstrate the efficacy of TARS in consistently hindering downstream distillation across teacher model scales (DeepSeek R1 Distill Qwen 7B, Gemma 3 12B), reasoning benchmarks (MMLU Pro, MATH, DeepMath), and student architectures (Llama 3.2 1B, Llama 3.2 3B, Gemma 3 1B, Qwen 3 1.7B) while preserving teacher performance.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.23238 [cs.CR] |
| (or arXiv:2604.23238v3 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2604.23238 arXiv-issued DOI via DataCite |
Submission history
From: Max Hartman [view email]
[v1]
Sat, 25 Apr 2026 10:22:09 UTC (505 KB)
[v2]
Fri, 8 May 2026 19:48:20 UTC (6,875 KB)
[v3]
Sat, 10 Oct 2026 13:08:20 UTC (8,801 KB)
来源：arXiv：cs.AI · arxiv.org