arXiv：cs.LG· El Mahdi Chayti, Alireza Mirrokni, Martin Jaggi·· 13 小时前
几何感知多目标优化：范数约束 Oracle 与熵权更新
Geometry-Aware Multi-Objective Optimization: Norm-Constrained Oracles with Entropic Weight Updates
AI 导读
该研究分析了一种单循环多目标方法，将范数约束的线性最小化 Oracle 与任务权重的熵更新相结合。对任意有限维范数对，在确定性设定下建立了 O(T^{-1/2}) 的遍历平稳率；在随机梯度与有界梯度假设下，单一混合梯度动量缓冲区达到 O(T^{-1/4}) 速率，其主导项不显式依赖任务数量。该分析将并行 MOON 的谱构造推广至一般范数，实验对比了现有多种多任务优化器。
正文
Abstract:Multiple-gradient descent is typically formulated in Euclidean geometry, whereas matrix-aware optimizers use directions induced by other norms. We analyze a single-loop multi-objective method combining a norm-constrained linear minimization oracle with entropic updates of task weights. For any finite-dimensional norm pair, we establish an $O(T^{-1/2})$ ergodic stationarity rate in the deterministic setting, deriving the required trajectory gradient bound from smoothness and lower-boundedness in the unregularized case. With stochastic gradients and a bounded-gradient assumption, one persistent blended-gradient momentum buffer yields an $O(T^{-1/4})$ rate whose leading term has no explicit dependence on the number of tasks. We also establish static regret for the inner weighting objective on arbitrary bounded gradient sequences and, with entropy regularization and a weight floor, iteration-wise inner-gap control after burn-in while retaining ergodic stationarity, conditional on a trajectory gradient bound. The optimal blended gradient can be nonunique in spectral and sign geometries, motivating a selection-independent inner gap. Our analysis extends the spectral construction studied by concurrent MOON to general norms. Experiments compare the method with existing multi-task optimizers and examine its oracle, weighting, and momentum choices.
| Subjects: | Optimization and Control (math.OC); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.13773 [math.OC] |
| (or arXiv:2610.13773v1 [math.OC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13773 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: El Mahdi Chayti [view email]
[v1]
Fri, 9 Oct 2026 15:55:53 UTC (77 KB)
来源：arXiv：cs.LG · arxiv.org