arXiv：cs.LG· Haorong Han, Jidong Yuan, Chixuan Wei, Yongqi Sun·· 4 小时前AI 评分30
Pioneer Student（PiS）：解耦 Teacher-Student 半监督学习的优化方案
Decoupled Optimization for Teacher-Student Semi-Supervised Learning via a Pioneer Student
AI 导读
研究者提出 Pioneer Student（PiS），一个在独立参数空间中运行的辅助分支，可定期将积累的知识回传给 Teacher-Student 模型，作为通用即插即用模块改善主流半监督学习（SSL）方法。该工作指出 T-S 框架存在参数耦合导致师生严格同步、以及标注与无标注损失梯度更新不一致使共享参数过早收敛到标注主导局部极小值两类优化问题。
正文
Abstract:Semi-supervised learning (SSL) relies on two core mechanisms: self-training under the Teacher-Student (T-S) framework and joint optimization of labeled and unlabeled losses. Despite their effectiveness, we find both mechanisms introduce distinct optimization pathologies. First, parameter coupling enforces strict synchronization between teacher and student, where strong regularization on the student degrades the teacher's fitting ability, thereby limiting the permissible generalization intensity. Second, the imbalance in gradient update consistency between labeled and unlabeled losses drives the shared parameters to prematurely converge to labeled-dominated local minima, creating a bottleneck for global optimization. To address both issues, we propose the Pioneer Student (PiS), an auxiliary branch that operates in an independent parameter space and periodically transfers accumulated knowledge back to the T-S model. Extensive experiments show that PiS is a universal plug-and-play module that consistently improves mainstream SSL methods.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.09609 [cs.LG] |
| (or arXiv:2610.09609v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09609 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Han Haorong [view email]
[v1]
Wed, 7 Oct 2026 07:54:40 UTC (750 KB)
来源：arXiv：cs.LG · arxiv.org