arXiv：cs.LG· Zhen Li·· 3 小时前
表征学习中的残差谱不稳定性
Residual spectral instabilities in representation learning
AI 导读
研究者将 VAE 中的逐维度后验坍缩表述为部分坍缩态周围的涨落理论，把负证据下界视为有效自由能，其二次展开的 Hessian 矩阵充当潜变量涨落的质量矩阵。坍缩方向构成不变涨落扇区，基于条件残差算子的精确质量谱给出局部重激活判据：当解码器方差低于残差谱上边缘时自由能下降，取等号即为临界。该判据在线性高斯 VAE 极限下可还原主成分阈值，数值延拓实验显示潜维度在这些谱临界点附近依次丢失。
正文
Abstract:Learned representations can lose latent degrees of freedom successively, suggesting a cascade of transitions whose underlying stability principle remains unclear. Here we formulate dimension-wise posterior collapse in variational autoencoder (VAE) as a fluctuation theory around partially collapsed states. Interpreting the negative evidence lower bound as an effective free energy, its quadratic expansion defines a Gaussian theory whose Hessian acts as a mass matrix for latent fluctuations. We show that the collapsed directions form an invariant fluctuation sector and derive its exact mass spectrum in terms of a conditional residual operator. A local reactivation direction lowers the free energy when the decoder variance falls below the residual spectral upper edge, with equality marking marginality. The criterion recovers principal component thresholds in the linear Gaussian VAE limit. Viewed in reverse along continuously connected branches, the reactivation boundary provides a local criterion for successive collapse. Numerical continuation experiments show successive loss of latent dimensions near these spectral marginalities. These results support a spectral cascade interpretation governed by residual information left unexplained by the surviving representation.
| Comments: | 19 pages, 3 figures |
| Subjects: | Machine Learning (cs.LG); Statistical Mechanics (cond-mat.stat-mech) |
| Cite as: | arXiv:2610.11257 [cs.LG] |
| (or arXiv:2610.11257v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11257 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhen Li [view email]
[v1]
Thu, 8 Oct 2026 05:09:43 UTC (108 KB)
来源：arXiv：cs.LG · arxiv.org