arXiv：cs.LG· Dengdi Sun, Xiaoya Zhou, Xiao Wang, Wanli Lyu, Jin Tang, Bin Luo·· 3 小时前AI 评分32
SPEAR：面向大规模 PDE 预训练的光谱解耦 MoE 神经算子
SPEAR: A Spectral-Disentangled MoE Neural Operator with Knowledge-Guided Expert Aggregation for Large-Scale PDE Pretraining
AI 导读
研究者提出 SPEAR，一种光谱解耦的 MoE 神经算子，通过将隐特征拆分为低频与高频分量，分别建模可迁移动力学与 PDE 专属模式，并用知识引导的专家聚合策略识别和合并相似专家。在 12 个 PDE 数据集及多个下游基准上，SPEAR 在预训练、微调和迁移学习任务中表现更优，同时将专家数量减少 50% 并保持或提升预测精度。
正文
Abstract:Large-scale pre-training has improved the generalization of neural operators across diverse PDEs. However, existing PDE foundation models still struggle with heterogeneous dynamics, where shared representations may cause knowledge interference, while mixture-of-experts (MoE) architectures suffer from increasing expert redundancy. We propose SPEAR, a spectral-disentangled MoE neural operator with knowledge-guided expert aggregation for large-scale PDE pre-training. SPEAR decouples latent features into low- and high-frequency components, enabling shared modeling of transferable dynamics and specialized learning of PDE-specific patterns. To address expert redundancy, we design a knowledge-guided expert aggregation strategy that measures expert similarity from dataset-specific learned knowledge and routing preferences, enabling the identification and consolidation of similar experts. Experiments on twelve PDE datasets and multiple downstream benchmarks demonstrate superior performance in pre-training, fine-tuning, and transfer learning. Furthermore, our aggregation strategy reduces the number of experts by 50\% while maintaining or improving prediction accuracy, achieving a balance between model efficiency and generalization for PDE foundation models.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.03265 [cs.LG] |
| (or arXiv:2610.03265v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03265 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiao Wang [view email]
[v1]
Fri, 2 Oct 2026 13:10:09 UTC (8,213 KB)
来源：arXiv：cs.LG · arxiv.org