arXiv：cs.AI· Israel Mason-Williams, Gabryel Mason-Williams, Helen Yannakoudakis·· 16 小时前
从函数中心视角重新审视平坦与尖锐极小值
A Function-Centric Perspective on Flat and Sharp Minima
AI 导读
一项被 TMLR 接收的研究从函数中心视角重新审视了极小值几何，认为平坦性不应被视为泛化的通用代理。研究覆盖单目标优化、合成非线性二分类到大规图像分类任务，发现正则化（如 weight decay、data augmentation、SAM）往往产生更尖锐的极小值，且可同时带来更好的泛化、校准、鲁棒性与函数一致性。
正文
Abstract:Flat minima are strongly associated with improved generalisation in deep neural networks. However, this connection has proven nuanced in recent studies, with both theoretical counterexamples and empirical exceptions emerging in the literature. In this paper, we revisit the role of sharpness in model performance and examine sharpness as a function-dependent property rather than as a direct indicator of poor generalisation. We conduct extensive empirical studies ranging from single-objective optimisation, synthetic non-linear binary classification tasks, to modern image classification tasks. In single-objective optimisation, we show that flatness and sharpness are relative to the function being learned: equally optimal solutions can exhibit markedly different local geometry. In synthetic non-linear binary classification tasks, we show that increasing decision-boundary tightness can increase sharpness even when models generalise perfectly, suggesting that sharpness is not reducible to memorisation alone. Finally, in large-scale experiments, we find that sharper minima often emerge when models are regularised (e.g., via weight decay, data augmentation, or SAM), and can coincide with better generalisation, calibration, robustness, and functional consistency. Our findings are consistent with a function-centric interpretation of minima geometry: flatness should not be treated as a universal proxy for generalisation, and sharper minima can reflect desirable inductive biases associated with improved predictive behaviour, calling for a function-centric reappraisal of minima geometry.
| Comments: | 45 pages, 48 tables, 17 figures, Accepted at Transaction on Machine Learning Research (TMLR) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2510.12451 [cs.LG] |
| (or arXiv:2510.12451v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2510.12451 arXiv-issued DOI via DataCite |
Submission history
From: Israel Mason-Williams [view email]
[v1]
Tue, 14 Oct 2025 12:33:14 UTC (2,373 KB)
[v2]
Wed, 15 Apr 2026 09:43:26 UTC (5,563 KB)
[v3]
Sun, 11 Oct 2026 17:28:39 UTC (5,569 KB)
来源：arXiv：cs.AI · arxiv.org