arXiv：cs.LG· Tobias Brock, Thomas Nagler·· 13 小时前
非平稳加权风险最小化的快速收敛率
Fast Rates for Nonstationary Weighted Risk Minimization
AI 导读
该研究分析了分布漂移下加权经验风险最小化的样本外预测误差，将超额风险分解为学习项与漂移项，并在混合条件下证明了学习误差的 oracle 不等式，该界对任意权重类一致成立。在平方损失下，作者给出了结构突变与平滑漂移过程的漂移误差显式界，并将其应用于线性模型、基函数近似和神经网络的回归问题，最优权重由学习与漂移的偏差-方差权衡决定。
正文
Abstract:Weighted empirical risk minimization is a common approach to prediction under distribution drift. This article studies its out-of-sample prediction error under nonstationarity. We provide a general decomposition of the excess risk into a learning term and an error term associated with distribution drift, and prove oracle inequalities for the learning error under mixing conditions. The learning bound holds uniformly over arbitrary weight classes and accounts for the effective sample size induced by the weight vector, the complexity of the weight and hypothesis classes, and potential data dependence. We also analyze the drift error in the square loss setting and provide explicit bounds for structural-break and smoothly drifting processes. We then combine the learning and drift analysis in (auto-) regression problems with linear models, basis approximations, and neural networks. In all cases, the optimal weights are characterized by a bias--variance trade-off between learning and drift, while minimax-optimal rates (up to logarithmic factors) are recovered when specialized to unweighted and stationary settings.
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST) |
| Cite as: | arXiv:2602.05742 [stat.ML] |
| (or arXiv:2602.05742v3 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2602.05742 arXiv-issued DOI via DataCite |
Submission history
From: Tobias Brock [view email]
[v1]
Thu, 5 Feb 2026 15:10:07 UTC (53 KB)
[v2]
Mon, 18 May 2026 09:02:06 UTC (54 KB)
[v3]
Fri, 9 Oct 2026 13:57:29 UTC (92 KB)
来源：arXiv：cs.LG · arxiv.org