arXiv：cs.AI· Shi-Yu Tian, Yong Liu, Ximan Liu, Tianrui Zhu, Sule Bai, Xinwei Yang, Ruiqi Liu, Kun-Yang Yu, Ming Yang, Zhi Zhou, Yu-Feng Li, Yipeng Sun, Gu Simiu·· 11 天前
Road MOPD：面向多教师同策略蒸馏的跨教师守护方法
Beyond Domain-Routed Supervision: Cross-Teacher Guarding for Multi-Teacher On-Policy Distillation
AI 导读
针对多教师同策略蒸馏（MOPD）中领域路由无法避免共享参数梯度冲突的问题，研究者提出 Cross-Teacher Guarding for MOPD（Road MOPD），用非路由教师作为跨域评判者，通过 token 级信用分配与梯度投影协调多教师监督。实验显示，Road MOPD 在高冲突场景下将弱势领域性能提升超过 5 个百分点，并在低冲突与三领域设置中改善整体表现。
正文 · 原文
Authors:Shi-Yu Tian, Yong Liu, Ximan Liu, Tianrui Zhu, Sule Bai, Xinwei Yang, Ruiqi Liu, Kun-Yang Yu, Ming Yang, Zhi Zhou, Yu-Feng Li, Yipeng Sun, Gu Simiu
Abstract:Multi-teacher on-policy distillation (MOPD) extends on-policy distillation (OPD) by routing domain-specific data to specialized teachers, integrating their capabilities through dense token-level supervision on student-generated trajectories. However, domain routing only distinguishes the supervision sources for different domains; updates from all domains still act on shared parameters. When teachers differ substantially in response style and domain expertise, supervision that is effective in isolation may still lead to persistent gradient conflicts and capability degradation during joint optimization. To address this challenge, we propose Cross-Teacher Guarding for MOPD (Road MOPD), which introduces non-routed teachers as auxiliary cross-domain critics on the same student trajectories and coordinates multi-teacher supervision through token-level credit assignment and gradient updates. Specifically, cross-teacher-guided credit assignment uses teacher agreement to strengthen token-level corrections jointly supported by the teachers and attenuate conflicting signals; preventive multi-teacher gradient projection uses auxiliary gradients from the same trajectories to constrain potentially conflicting updates and adaptively controls projection strength to retain target-domain learning signals. In experiments, Road MOPD mitigates degradation in the disadvantaged domain, gains over 5 percentage points under high conflict, and improves overall performance in low-conflict and three-domain settings.
| Comments: | 30 pages, 9 figures, 10 tables, including references and appendices |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.13521 [cs.AI] |
| (or arXiv:2610.13521v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13521 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shi-Yu Tian [view email]
[v1]
Fri, 9 Oct 2026 12:41:51 UTC (1,102 KB)
来源：arXiv：cs.AI · arxiv.org