arXiv：cs.CL· Zhenhe Wu, Jian Yang, Yaping Jin, Xinlai Kang, Jie Hu, Hanting Chen·· 16 小时前
PA1-OPD：面向语言模型预训练的一步前缀增强同策略蒸馏
PA1-OPD: One-Step Prefix-Augmented On-Policy Distillation for Language Model Pretraining
AI 导读
PA1-OPD 在语料前缀的知识蒸馏基础上，对每个选定前缀用学生模型采样一步后引入教师反馈，扩展监督状态的同时保留下一 token 预测与语料路径蒸馏。
正文
Abstract:Knowledge distillation (KD) improves language-model pretraining by matching teacher distributions at corpus prefixes, but provides no direct supervision at contexts induced by student predictions. We introduce PA1-OPD, which augments corpus-path distillation with teacher feedback after one student-sampled transition per selected prefix. This expands the supervised states while retaining next-token prediction and corpus-path KD. Data-anchored roots support parallel sampling, and differentiable tree attention shares prefix computation without changing inference. In two continued-pretraining settings, full coverage improves held-out cross-entropy over KD across C4, PG-19, and Pile at every evaluated checkpoint. In Stage2, selecting the 10% of roots with highest student surprisal adds 48.85% to KD's Macro cross-entropy gain over next-token training. Gains remain when branch supervision excludes successor states already on the corpus path, supporting the value of feedback beyond observed contexts. Additional depth or breadth improves aggregate results at unchanged per-term supervision weight, with benefits varying by corpus. These findings support local student-induced supervision as a complement to pretraining distillation.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19601 [cs.CL] |
| (or arXiv:2610.19601v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19601 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhenhe Wu [view email]
[v1]
Wed, 14 Oct 2026 05:23:00 UTC (213 KB)
来源：arXiv：cs.CL · arxiv.org