arXiv：cs.LG（机器学习，全量分类）· Simone Brivio, Nicola Rares Franco·· 15 小时前AI 评分35
从 Eckart-Young-Schmidt 视角看深度对称自编码器
Deep Symmetric Autoencoders from the Eckart-Young-Schmidt Perspective
AI 导读
研究者对深度对称自编码器提出形式化分类，并证明带正交约束的对称自编码器重建误差可用 Eckart-Young-Schmidt（EYS）定理刻画。基于此提出 EYS 初始化策略，通过迭代应用奇异值分解（SVD）实现，并在数值实验中与传统深度自编码器对比，讨论了模型设计与初始化的重要性。
正文
Abstract:Deep autoencoders have become a fundamental tool in various machine learning applications, ranging from dimensionality reduction and reduced order modeling of partial differential equations to anomaly detection and neural machine translation. Despite their empirical success, a solid theoretical foundation for their expressiveness remains elusive, particularly when compared to classical projection-based techniques. In this work, we aim to take a step forward in this direction by presenting a comprehensive analysis of what we refer to as symmetric autoencoders, a broad class of deep learning architectures that have surfaced frequently in the recent literature. Specifically, we introduce a formal distinction between different classes of symmetric architectures, analyzing their strengths and limitations from a mathematical perspective. For instance, we show that the reconstruction error of symmetric autoencoders with orthonormality constraints can be understood by leveraging the well-renowned Eckart-Young-Schmidt (EYS) theorem. As a byproduct of our analysis, we end up developing the EYS initialization strategy for symmetric autoencoders, which is based on an iterated application of the Singular Value Decomposition (SVD). To validate our findings, we conduct a series of numerical experiments where we benchmark our proposal against conventional deep autoencoders, discussing the importance of model design and initialization.
| Comments: | 32 pages, 12 figures |
| Subjects: | Numerical Analysis (math.NA); Machine Learning (cs.LG) |
| MSC classes: | 68T07, 47N40 |
| Cite as: | arXiv:2506.11641 [math.NA] |
| (or arXiv:2506.11641v2 [math.NA] for this version) | |
| https://doi.org/10.48550/arXiv.2506.11641 arXiv-issued DOI via DataCite |
Submission history
From: Simone Brivio [view email]
[v1]
Fri, 13 Jun 2025 10:12:34 UTC (1,404 KB)
[v2]
Wed, 30 Sep 2026 21:25:08 UTC (2,153 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org