arXiv：cs.LG（机器学习，全量分类）· Zekai Chen, Xun Wu, Hailin Zhang, Xunkai Li, Yu Liu, Kairui Yang, Muyan Huang, Xuaner Chen, Rong-Hua Li, Guoren Wang·· 14 小时前AI 评分24
FedCORE：在联邦多模态图基础模型中耦合感知与推理
Coupling Perception and Reasoning in Federated Multimodal Graph Foundation Models
AI 导读
针对联邦多模态图基础模型中多模态 Encoder 冻结、仅更新 GNN 导致感知与推理适配割裂的问题，研究者提出 FedCORE，用共享低维隐状态表示 Encoder 与 GNN 的更新，并在该状态空间中联合优化与联邦演化。实验显示 FedCORE 将 Encoder–GNN 配对差距从 30.6 降至 5.9，相比独立联合适配减少 80.7%。
正文
Abstract:Federated multimodal graph foundation models (GFMs) aim to adapt pretrained multimodal models to decentralized graph data, where each client owns a private multimodal graph and cannot share raw information. These models typically combine a multimodal Encoder that extracts semantic evidence from heterogeneous modalities and a graph neural network (GNN) that performs relational reasoning over graph structures. However, existing federated GFM adaptation methods mainly update graph-side modules while keeping the multimodal Encoder frozen, limiting adaptation to \emph{how information is propagated} while fixing \emph{what information is extracted}. Through empirical studies, we reveal that Encoder and GNN adaptations are not independent: Encoder adaptation is affected by graph relations, while cross-client module swapping reveals substantial pairing sensitivity between separately parameterized Encoder and GNN updates. Motivated by this observation, we propose \textbf{FedCORE}, a federated adaptation framework that represents Encoder and GNN updates through a shared low-dimensional latent state. FedCORE jointly optimizes this core from multimodal and structural signals and performs federated evolution directly in the shared state space, preserving compatibility between perception and reasoning adaptations. Extensive experiments demonstrate that FedCORE reduces the Encoder--GNN pairing gap from $30.6$ to $5.9$, corresponding to an $80.7\%$ reduction over independent joint adaptation.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.00277 [cs.LG] |
| (or arXiv:2610.00277v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00277 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zekai Chen [view email]
[v1]
Thu, 24 Sep 2026 16:42:36 UTC (2,119 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org