arXiv：cs.LG· Yufeng Yang, Fangning Zhuo, Ziyi Chen, Heng Huang, Yi Zhou·· 7 小时前AI 评分33
从双循环跟踪到梯度裁剪：可证明更快的分布式鲁棒多目标优化
From Dual Tracking to Clipping: Provably Faster Distributionally Robust Multi-Objective Optimization
AI 导读
研究者提出分布式鲁棒多目标优化（DR-MOO），在各目标各自的最坏分布下最小化多个目标，并给出帕累托型解概念与可证明保证的多梯度下降算法（MGDA）。基于拉格朗日对偶重构的双循环 MGDA 达到 ε-Pareto 稳定点的样本复杂度为 O(ε^-8)；结合大批量采样与梯度裁剪的单循环 double-clip MGDA 将复杂度降至 O(ε^-4)。
正文
Abstract:Multi-objective optimization (MOO) has received growing attention in applications that require learning under multiple criteria. However, most existing MOO formulations do not explicitly account for distributional shifts in the data. We introduce distributionally robust multi-objective optimization (DR-MOO), which minimizes multiple objectives under their respective worst-case distributions. We propose Pareto-type solution concepts for DR-MOO and develop multi-gradient descent algorithms (MGDA) with provable guarantees. Leveraging a Lagrangian dual reformulation, we first design a double-loop MGDA that uses an inner loop to estimate dual variables and achieves a total sample complexity $\mathcal{O}(\epsilon^{-8})$ for reaching an $\epsilon$-Pareto-stationary point. To further improve convergence, we combine large-batch sampling with gradient clipping to accommodate generalized smoothness and control bias in stochastic preference updates, eliminating the need for double sampling. This yields a single-loop double-clip MGDA with substantially improved sample complexity $\mathcal{O}(\epsilon^{-4})$. Our theory applies to nonconvex problems without requiring uniformly bounded gradients of the dual objectives. Experiments demonstrate that our methods are competitive with state-of-the-art MGDA baselines.
| Comments: | 47 pages |
| Subjects: | Machine Learning (cs.LG); Optimization and Control (math.OC) |
| Cite as: | arXiv:2605.05660 [cs.LG] |
| (or arXiv:2605.05660v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.05660 arXiv-issued DOI via DataCite |
Submission history
From: Yufeng Yang [view email]
[v1]
Thu, 7 May 2026 04:24:17 UTC (3,689 KB)
[v2]
Sat, 26 Sep 2026 02:46:50 UTC (5,236 KB)
[v3]
Mon, 5 Oct 2026 22:53:27 UTC (5,236 KB)
来源：arXiv：cs.LG · arxiv.org