arXiv：cs.LG· Eugene Belilovsky, Stefan Horoi, Edouard Oyallon, Benjamin Th\'erien·· 4 小时前
内层优化器如何影响 LLM 副本扩展
How Inner Optimizers Shape Replica Scaling
AI 导读
研究分析了内层优化器对 LLM 副本扩展（replica scaling）性能的影响，指出伪梯度的协方差结构和 batch size 对伪梯度噪声尺度的影响是关键因素。理论与实验表明，基于白化的预条件（whitening-based pre-conditioning）是理想的内层优化器特性，在 8-96 个副本范围内实现强副本扩展，且随参数和数据规模增大，相对 DP 的性能持续改善。
正文
Abstract:Training large language models (LLMs) across geographically distributed and poorly interconnected compute pools or data centers has become a topic of significant interest. This has led to increased study of local optimization methods such as DiLoco for LLMs, especially those that use local (or inner) adaptive or preconditioned optimizers, which are critical for high-performance LLM training. However, a central challenge remains: it is well known that increasing the number of model replicas (replica scaling) while fixing the total training budget leads to monotonically decreasing performance. Existing work has empirically demonstrated that the choice of inner optimizer (Muon v.s. AdamW) can significantly improve this trend, but offers only limited explanation of the performance difference. In this work, we consider a range of inner-optimizers and identify two important factors that impact replica scaling: (1) the covariance structure of the pseudo-gradients and (2) the impact of batch size on the pseudogradient's noise scale. Based on these insights, we theoretically and empirically demonstrate that whitening-based pre-conditioning is a powerful inner-optimizer desideratum. Our empirical evaluation extends the results to reveal that whitening-based inner-optimizers achieve strong replica scaling (for 8-96 replicas), with performance relative to DP improving as parameters and data are scaled.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.19064 [cs.LG] |
| (or arXiv:2610.19064v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19064 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Eugene Belilovsky [view email]
[v1]
Tue, 13 Oct 2026 19:00:27 UTC (226 KB)
来源：arXiv：cs.LG · arxiv.org