arXiv：cs.AI· Yutong Chen, Guangfu Guo, Zhichao Xu, Kunpeng Liu·· 5 小时前
DualOPSD：在 On-Policy 自蒸馏中闭合反馈回路
DualOPSD: Closing the Feedback Loop in On-Policy Self-Distillation
AI 导读
DualOPSD 提出一种耦合的教师-学生交替更新机制，让学生保留 OPSD 的截断前向 KL 目标，特权教师则在每步学生更新后自适应调整，并以冻结初始模型为残差目标保留特权上下文效果。
正文
Abstract:On-policy self-distillation (OPSD) combines student generated trajectories with dense supervision from a privileged teacher of the same model. Its pointwise clipping protects the student from locally incompatible teacher directions, but the fixed teacher receives no feedback about the supervision the student rejects, which limits effective learning from the teacher. We introduce DualOPSD, which alternates updates in a coupled teacher-student system. The student retains OPSD's clipped forward KL objective. The privileged teacher adapts to the updated student after each student step. A residual target based on the frozen initial model preserves the effect of privileged context during teacher adaptation. Across Qwen3-8B/4B/1.7B and three math benchmarks, DualOPSD improves avg@12 accuracy over OPSD by 9.83 percentage points. Further diagnostic analyses show lower bidirectional KL, less persistent mismatch, and retained sensitivity to privileged reference. Together, these results demonstrate that DualOPSD can be a more effective form of privileged self-distillation. Code is available at: this https URL.
| Comments: | preprint |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.26019 [cs.LG] |
| (or arXiv:2608.26019v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2608.26019 arXiv-issued DOI via DataCite |
Submission history
From: Yutong Chen [view email]
[v1]
Wed, 26 Aug 2026 17:01:21 UTC (171 KB)
[v2]
Tue, 13 Oct 2026 17:08:56 UTC (356 KB)
来源：arXiv：cs.AI · arxiv.org