arXiv：cs.LG· El Mahdi Chayti, Mohamad Fakhouri·· 13 小时前
混合动量 Frank-Wolfe 实现随机复合优化
Stochastic Compositional Optimization via Hybrid Momentum Frank--Wolfe
AI 导读
研究者提出 Hybrid Momentum Stochastic Frank-Wolfe，一种用于外层函数 F 的无投影随机复合优化方法，F 联合凸且 Lipschitz 但未必可微，可覆盖 maxima、CVaR、范数等目标。
正文
Abstract:Stochastic compositional optimization minimizes $F(f(x), x)$ over a compact convex set when the inner map $f$ is available only through noisy samples. We propose Hybrid Momentum Stochastic Frank--Wolfe, a projection-free method for outer functions $F$ that are jointly convex and Lipschitz but not necessarily differentiable, such as maxima, CVaR, and norms. The method tracks the Jacobian and the value of $f$ with momentum, using a Taylor correction for the value, and passes the tracked linearization to a generalized linear minimization oracle. Under finite-variance noise, it attains an expected generalized Frank--Wolfe gap of order $K^{-1/4}$ for non-convex objectives and, when the linearized models minorize the objective, a last-iterate primal error of order $K^{-1/3}$. The analysis extends to heavy-tailed noise without clipping, with separate moment orders for values and Jacobians. With an exact oracle, the method reduces to the deterministic method of Vladarean et al. (2023).
| Subjects: | Optimization and Control (math.OC); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.15350 [math.OC] |
| (or arXiv:2605.15350v2 [math.OC] for this version) | |
| https://doi.org/10.48550/arXiv.2605.15350 arXiv-issued DOI via DataCite |
Submission history
From: El Mahdi Chayti [view email]
[v1]
Thu, 14 May 2026 19:20:22 UTC (60 KB)
[v2]
Fri, 9 Oct 2026 17:27:46 UTC (7,389 KB)
来源：arXiv：cs.LG · arxiv.org