arXiv：cs.LG（机器学习，全量分类）· Wei Jiang, Rui Yan, Sifan Yang, Yuanyu Wan, Lijun Zhang, Zechao Li·· 17 小时前AI 评分32
随机多级复合优化的最优动量方法
Optimal Momentum Methods for Stochastic Multilevel Compositional Optimization
AI 导读
该论文研究目标为多个光滑非凸函数嵌套复合的随机多级优化问题，提出基于动量的估计器配合 mini-batch 追踪各层函数值，在无需平均光滑性假设下达到 O(ε^-4) 的最优样本复杂度。通过归一化技术，无需问题相关常数即可设置超参数；进一步提出无 batch 方法，结合一阶近似与裁剪技术估计函数值。在风险规避投资组合优化和分层倾斜经验风险最小化实验上验证了方法有效性。
正文
Abstract:This paper investigates stochastic multi-level optimization where the objective is a nested composition of several smooth non-convex functions. We assume that only stochastic estimates of the gradient and function values for each level are accessible. Consequently, obtaining an accurate estimate of the overall gradient is challenging due to the nested structure. To address this, we employ a momentum-based estimator with mini-batches to track the function values of each level, which are subsequently used to construct momentum gradient estimators. We establish an optimal sample complexity of $\mathcal{O}(\epsilon^{-4})$ for finding an $\epsilon$-stationary point, avoiding the stronger average smoothness assumption commonly relied upon in prior literature. Furthermore, by employing a normalization technique, we attain the same rate without requiring problem-dependent constants to set hyperparameters. To achieve the optimal rate without mini-batches, we further develop a batch-free method that incorporates a first-order approximation and a clipping technique for function value estimation. Finally, we validate the effectiveness of our proposed methods through experiments on risk-averse portfolio optimization and hierarchical tilted empirical risk minimization.
| Subjects: | Optimization and Control (math.OC); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.01572 [math.OC] |
| (or arXiv:2610.01572v1 [math.OC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.01572 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wei Jiang [view email]
[v1]
Thu, 1 Oct 2026 12:32:40 UTC (3,762 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org