arXiv：cs.LG· Zhongjing Gu, Fengqiang Wan, Yiming Cui, Yufa Feng, Yang Yang·· 3 小时前AI 评分35
FGMO：用函数空间进度信号平衡多模态学习
Balancing Multimodal Learning via Functional Progress
AI 导读
针对多模态学习中单一模态主导联合优化的问题，研究者提出 Function-Space Guided Multimodal Optimization（FGMO），用函数空间进度信号评估各模态优化进展并协调优化。
正文
Abstract:Multimodal learning often suffers from modality imbalance, where the joint optimization process is dominated by a single modality. Existing methods typically estimate modality imbalance from score disparities derived from prediction uncertainty or optimization statistics. However, due to distinct prediction uncertainty and learning dynamics across modalities, direct comparison of such scores may misinterpret intrinsic modality differences as progress gaps, leading to biased imbalance estimation. In this paper, we propose Function-Space Guided Multimodal Optimization (FGMO), which leverages a function-space progress signal to assess modality-wise optimization progress and coordinate optimization across modalities to alleviate modality imbalance. Specifically, we introduce Functional Progress Estimation (FPE) to measure each modality's update-induced function-space response and calibrate it against a loss-aligned unimodal reference, producing a comparable progress signal. Based on this signal, Functional Response Control (FRC) redistributes modality-level function-space budgets and realizes the target responses through tensor-wise learning-rate adjustment. Theoretical analysis establishes a one-step target-contraction property of FRC under bounded controller-state mismatch, and extensive experiments demonstrate the effectiveness of FGMO across multiple multimodal benchmarks.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.03035 [cs.LG] |
| (or arXiv:2610.03035v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03035 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhongjing Gu [view email]
[v1]
Fri, 2 Oct 2026 09:17:58 UTC (1,082 KB)
来源：arXiv：cs.LG · arxiv.org