arXiv：cs.LG· Zhewei Chen, Hao Zhu, Jiaojiao Jiang, Ahad N. Zehmakan·· 4 小时前AI 评分32
G^2MLP：从 GNN 到 MLP 的图几何知识蒸馏
Distilling Graph Geometry: Knowledge Gap from GNNs to MLPs
AI 导读
研究者提出 Graph Geometry-aware MLP（G^2MLP），一个由 Ollivier-Ricci 曲率引导的训练期蒸馏框架，用于 GNN 到 MLP 的知识蒸馏。
正文
Abstract:GNN-to-MLP distillation aims to retain the predictive accuracy of a message-passing teacher while deploying a graph-free MLP at inference. Existing methods mainly transfer node-wise predictions or use confidence-based reweighting, but they do not specify where the student should preserve the teacher's graph-induced geometry. We show that this omission leads to two spectral failure modes in the student's representation space. On sparse graphs, the student suffers from spectral underfit, missing high-energy teacher directions concentrated near boundary regions. On dense graphs, it suffers from spectral overfit, retaining spurious directions that the teacher has collapsed through aggregation. Motivated by an energy-weighted teacher-student alignment objective, we propose Graph Geometry-aware MLP (G^2MLP), a training-time distillation framework guided by Ollivier-Ricci curvature. Curvature identifies where the two spectral errors concentrate and is used to allocate supervision between prediction-level and representation-level alignment. The deployed model remains a standard MLP and requires no graph access at inference. Across node-classification benchmarks, G^2MLP consistently improves over graph-free distillation baselines, reduces the teacher-student rank gap in both regimes, and transfers without architectural changes to Graph Transformer teachers and link prediction.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.10520 [cs.LG] |
| (or arXiv:2610.10520v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10520 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhewei Chen [view email]
[v1]
Wed, 7 Oct 2026 17:56:01 UTC (2,155 KB)
来源：arXiv：cs.LG · arxiv.org