arXiv：cs.LG· Vera Kurkova, Marcello Sanguineti·· 3 小时前
有限 VC 维网络的是与非：逼近能力与学习一致性之间的权衡
Networks with Finite VC Dimension: Pro and Contra
AI 导读
一项理论研究比较了有限 VC 维对神经网络逼近能力与学习一致性的不同影响：有限 VC 维有利于经验误差的一致收敛，却未必有利于逼近服从概率分布的函数。基于高维几何的测度集中性质，论文证明，当网络输入输出函数集具有有限 VC 维且处理大规模数据集时，逼近误差与经验误差均近似确定性地表现。
正文
Abstract:Approximation and learning of classifiers of large data sets by neural networks in terms of high-dimensional geometry and statistical learning theory are investigated. The influence of the VC dimension of sets of input-output functions of networks on approximation capabilities is compared with its influence on consistency in learning from samples of data. It is shown that, whereas finite VC dimension is desirable for uniform convergence of empirical errors, it may not be desirable for approximation of functions drawn from a probability distribution modeling the likelihood that they occur in a given type of application. Based on the concentration-of-measure properties of high dimensional geometry, it is proven that both errors in approximation and empirical errors behave almost deterministically for networks implementing sets of input-output functions with finite VC dimensions in processing large data sets. Practical limitations of the universal approximation property, the trade-offs between the accuracy of approximation and consistency in learning from data, and the influence of depth of networks with ReLU units on their accuracy and consistency are discussed.
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG) |
| Cite as: | arXiv:2502.02679 [stat.ML] |
| (or arXiv:2502.02679v3 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2502.02679 arXiv-issued DOI via DataCite |
Submission history
From: Marcello Sanguineti [view email]
[v1]
Tue, 4 Feb 2025 19:44:14 UTC (27 KB)
[v2]
Sat, 15 Nov 2025 15:42:46 UTC (27 KB)
[v3]
Thu, 8 Oct 2026 09:37:28 UTC (431 KB)
来源：arXiv：cs.LG · arxiv.org