arXiv：cs.AI· Jiaqi Tian, Ting Long·· 11 小时前
Representation Geometry Regularization 缓解 LLM 自改进中的能力退化
Mitigating Capability Degradation in LLM Self-Improvement via Representation Geometry Regularization
AI 导读
针对 On-Policy Self-distillation（OPSD）在提升 LLM 整体表现的同时会让早期已答对的问题逐渐答错的能力退化现象，研究者提出 Representation Geometry Regularization（RGR）。
正文
Abstract:On-Policy Self-distillation is a self-improvement paradigm that enables large language models (LLMs) to improve their capabilities by leveraging the same model as both teacher and student under different conditioning contexts. However, we find that OPSD methods can progressively degrade previously acquired capabilities: queries that are correctly responded at earlier training stages may become incorrectly responded as optimization proceeds, even the overall performance continues to improve. Further analysis reveals that this degradation coincides with a deterioration of the geometry of hidden representations, characterized by reduced intra-cluster compactness and diminished inter-cluster separation. Motivated by this observation, we propose Representation Geometry Regularization (RGR), which preserves representation geometry during post-training by encouraging compactness within cluster and separation across cluster. RGR can be seamlessly integrated into existing OPSD methods without altering their core optimization procedures. Extensive experiments on multiple OPSD methods and benchmarks show that RGR generally alleviates capability degradation while retaining capability improvements. Our code is available in this https URL.
| Comments: | 15 pages, 5 figures |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.12855 [cs.AI] |
| (or arXiv:2610.12855v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12855 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiaqi Tian [view email]
[v1]
Fri, 9 Oct 2026 01:28:50 UTC (1,390 KB)
来源：arXiv：cs.AI · arxiv.org