arXiv：cs.LG· Adam Piaseczny, Md Kamran Chowdhury Shisher, Shiqiang Wang, Christopher G. Brinton·· 3 小时前AI 评分33
选择性状态空间模型的分布式学习：架构感知的收敛性分析
Distributed Learning with Selective State Space Models: Architecture-Aware Convergence Analysis
AI 导读
研究针对 Mamba2 等选择性 SSM 在分布式学习中行为不明的问题，推导了单层与多层选择性 SSM 的架构感知梯度与平滑性边界，以及 FedAvg 和 FedProx 的收敛边界，刻画循环稳定性、输入相关离散化和状态投影范数对联邦优化的影响。团队在教师 SSM 生成序列上数值验证了单层边界，并在六个文本域上对比九种联邦学习算法在 Mamba2 语言建模上的表现。
正文
Abstract:Modern state space models (SSMs), such as Mamba2, provide a compelling alternative to transformers by combining linear-time sequence modeling with recurrent state-space dynamics. However, the behavior of SSMs in distributed learning settings remains poorly understood. In particular, the existing standard federated learning methods are largely architecture-agnostic, and do not account for the stability, selectivity, and state-space parameterization that characterize modern selective SSMs. To address this, we derive architecture-aware gradient and smoothness bounds for single- and multi-layer selective SSMs, and convergence bounds for FedAvg and FedProx, characterizing how recurrent stability, input-dependent discretization, and state projection norms affect federated optimization. We then numerically validate the single-layer bounds on sequences generated by a teacher SSM, using a learner that follows the analyzed recurrence. We use this analysis to formulate expectations about the effects of local training and client heterogeneity, and examine these expectations by comparing nine federated learning algorithms on Mamba2 language modeling across six text domains. These experiments illustrate how SSM-specific bounds can provide a basis for interpreting the behavior of practical federated learning algorithms.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC) |
| Cite as: | arXiv:2610.02659 [cs.LG] |
| (or arXiv:2610.02659v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.02659 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Adam Piaseczny [view email]
[v1]
Fri, 2 Oct 2026 01:28:31 UTC (1,321 KB)
来源：arXiv：cs.LG · arxiv.org