MAdam：面向多目标优化的度量感知 Adam 封装器

View PDF HTML (experimental)

Abstract:Multi-objective optimization (MOO) underlies many machine learning problems, yet MOO solvers across the loss-balancing, gradient-balancing, and Pareto-based families almost universally hand their reconciled directions to Adam~\citep{kingma2015adam}. We show this coupling introduces two systematic gaps between the solver's intent and the optimizer's execution. The first is a weighting mismatch: Adam's second-moment denominator entangles the time-varying preference vector with gradient statistics, marginalizing the preference into a history average and collapsing distinct Pareto trade-offs toward a near-uniform mixture. The second is a geometric mismatch: Adam's adaptive metric distorts the Euclidean geometry MOO solvers assume, turning aligned objectives into apparent conflicts. To resolve both jointly, we introduce MAdam (Metric-Aware Multi-Objective Adam), a drop-in wrapper that leaves both solver and optimizer unchanged. MAdam preconditions the reconciled direction by the preference-conditioned curvature of the scalarized objective; on this whitened input, Adam's second moment collapses to identity, so the realized update is governed by the preference-conditioned metric. Across multi-task learning, Pareto-front recovery, physics-informed neural networks, and medical imaging, MAdam improves over Adam in aggregate for every solver family. Our code is available at this https URL.

Comments:	NeurIPS 2026
Subjects:	Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
Cite as:	arXiv:2606.03904 [cs.LG]
	(or arXiv:2606.03904v2 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2606.03904 arXiv-issued DOI via DataCite

Submission history

From: Fengbei Liu [view email]
[v1] Tue, 2 Jun 2026 17:00:15 UTC (7,470 KB)
[v2] Tue, 6 Oct 2026 20:07:49 UTC (7,476 KB)