arXiv：cs.LG（机器学习，全量分类）· Wei Jiang, Yibo Wang, Wenhao Yang, Rui Yan, Lijun Zhang, Zechao Li·· 15 小时前AI 评分30
STORM 在不同几何条件下的收敛性分析
Convergence Analysis of STORM Under Different Geometries
AI 导读
研究在去除平均平滑假设后 STORM 的收敛性，通过设计辅助序列与 STORM 更新对比，证明其在非凸目标下仍达到标准平滑条件下的最优速率 O(T^{-1/4})。在凸和 λ-强凸目标下，分别得到 O(σR/√T) 和 O(σ²/(λT)) 的最优平均与末次迭代界，且所有结果均使用同一 STORM 递归与不同超参数选择。
正文
Abstract:Stochastic recursive momentum (STORM) achieves fast convergence for nonconvex optimization via the variance reduction effect, but existing analyses rely on the strong average smoothness assumption. In this paper, we study the convergence of STORM for different objectives without average smoothness. We first revisit the results under average smoothness, obtaining the $O(T^{-1/3})$ bound for nonconvex objectives and the $O(\sigma^2/(\mu T))$ bound for last-iterate output under the $\mu$-Polyak--Łojasiewicz~(PL) condition. Without average smoothness, we design an auxiliary sequence and compare the STORM update with it in the analysis. With the help of this sequence, we prove that STORM still attains an $O(T^{-1/4})$ rate for nonconvex objectives, which is optimal under standard smoothness. For convex and $\lambda$-strongly convex objectives, we further prove averaged and last-iterate bounds with optimal rates of $O(\sigma R/\sqrt T)$ and $O(\sigma^2/(\lambda T))$, respectively. All the obtained results use the same STORM recursion with different hyperparameter choices.
| Subjects: | Optimization and Control (math.OC); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.01599 [math.OC] |
| (or arXiv:2610.01599v1 [math.OC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.01599 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wei Jiang [view email]
[v1]
Thu, 1 Oct 2026 12:46:42 UTC (243 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org