arXiv：cs.LG· Zong Shang, Tomoya Wakayama, Guillaume Lecu\'e, Taiji Suzuki·· 4 小时前AI 评分33
平均场 Langevin 动力学中统计特征学习的几何理论
The Geometry of Statistical Feature Learning in Mean-Field Langevin Dynamics
AI 导读
研究者提出监督回归中统计特征学习的几何框架，用底-纤维分解刻画训练产生的特征侧几何与学习得到的特征空间，并在球面平均场 Langevin 动力学上证明了该性质。在高斯多指标模型中，低温平稳分布集中于隐藏指标并形成多峰结构，温度 λ≍1 处出现急剧转变；高斯单指标模型的平稳测度则呈现由奇偶性决定的集中现象，学习到的特征空间可实现 d/N 与 Md/N 量级的学习速率。
正文
Abstract:We introduce a geometric formulation of statistical feature learning for supervised regression. Feature learning is defined through a base--fiber decomposition: the base is the feature-side geometry produced by training, and the fiber is the learned feature space where estimation is performed. We prove this property for spherical mean-field Langevin dynamics, viewed as the Wasserstein gradient flow of a negative entropy-regularized empirical risk. In Gaussian multi-index models, the low-temperature stationary distribution concentrates near the hidden indices, forms a multi-spike structure, and yields parameter recovery with high probability, even though negative entropy regularization penalizes concentration. This concentration has a sharp transition at temperature $\lambda\asymp 1$. In Gaussian single-index models, the stationary measure satisfies a concentration property, with parity determining whether it lives on $S_2^{d-1}$ or $\mathbb{RP}^{d-1}$. The induced learned feature space aligns the regression signal and yields rates $d/N$ and $Md/N$, up to logarithmic factors.
| Subjects: | Statistics Theory (math.ST); Machine Learning (cs.LG); Machine Learning (stat.ML) |
| Cite as: | arXiv:2606.31429 [math.ST] |
| (or arXiv:2606.31429v2 [math.ST] for this version) | |
| https://doi.org/10.48550/arXiv.2606.31429 arXiv-issued DOI via DataCite |
Submission history
From: Zong Shang [view email]
[v1]
Tue, 30 Jun 2026 09:54:32 UTC (106 KB)
[v2]
Thu, 30 Jul 2026 11:25:46 UTC (117 KB)
来源：arXiv：cs.LG · arxiv.org