重尾数据下经验风险最小化的渐近分析

View PDF HTML (experimental)

Abstract:Many real-world datasets exhibit unusually large values far more frequently than predicted by Gaussian models. Heavy-tailed distributions capture this behavior, yet evaluating learning performance under them remains challenging because rare, large feature entries retain non-vanishing effects even in high dimensions. Even in the canonical setting of empirical risk minimization for linear regression with entry-wise i.i.d. symmetric $\alpha$-stable data, a precise asymptotic characterization of prediction has been lacking. In this work, we introduce a functional order parameter that describes the random effective problem associated with each coefficient. Using the replica method, we fully characterize the generalization error in the proportional high-dimensional limit where the sample size and feature dimension diverge at a fixed ratio. Additionally, this analysis establishes a heavy-tail universality law, scaling laws relating typical errors to prediction reliability, and the Bayes-optimal prediction error. In addition to characterizing the effects of extreme entries on the learning process, our method applies broadly to other systems with persistent local heterogeneity.

Subjects:	Disordered Systems and Neural Networks (cond-mat.dis-nn); Machine Learning (cs.LG); Statistics Theory (math.ST); Machine Learning (stat.ML)
Cite as:	arXiv:2610.07637 [cond-mat.dis-nn]
	(or arXiv:2610.07637v1 [cond-mat.dis-nn] for this version)
	https://doi.org/10.48550/arXiv.2610.07637 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Kaito Takanami [view email]
[v1] Tue, 6 Oct 2026 02:27:07 UTC (145 KB)