arXiv：cs.LG· Kevin Han Huang, Haoyu Ye, Somak Laha, Morgane Austern·· 16 小时前
设计矩阵的零模式驱动过参数化回归中的多重下降
The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression
AI 导读
一项理论研究放宽了过参数化线性回归中协变量独立且协方差矩阵非退化的假设，在消失岭回归机制下推导出预测风险的确定性等价式。研究表明，协方差矩阵的退化与变量间的依赖关系会导致多重下降，并刻画了相应峰值出现的位置。证明过程采用方差谱的新图表示，通过最大匹配与相关二分图的 Dulmage-Mendelsohn 分解，识别出方差变为奇异的配置。
正文
Abstract:Over-parameterized linear regression has been widely studied over the last decade. However, most existing works assume that the covariates are independent and that their covariance matrices are non-degenerate. In this paper, we relax both assumptions and derive deterministic equivalents for the prediction risk in a vanishing-ridge regime. We show that degeneracy of the covariance matrices and dependence can lead to multiple descent, and characterize where the corresponding peaks can occur. Our proofs use a novel graph representation of the variance profile. We show that maximum matchings and the Dulmage--Mendelsohn decomposition of the associated bipartite graph identify the configurations at which the variance becomes singular.
| Subjects: | Statistics Theory (math.ST); Machine Learning (cs.LG); Machine Learning (stat.ML) |
| Cite as: | arXiv:2607.24041 [math.ST] |
| (or arXiv:2607.24041v3 [math.ST] for this version) | |
| https://doi.org/10.48550/arXiv.2607.24041 arXiv-issued DOI via DataCite |
Submission history
From: Haoyu Ye [view email]
[v1]
Mon, 27 Jul 2026 06:30:49 UTC (1,047 KB)
[v2]
Thu, 10 Sep 2026 17:27:30 UTC (1,024 KB)
[v3]
Wed, 14 Oct 2026 17:50:43 UTC (1,011 KB)
来源：arXiv：cs.LG · arxiv.org