arXiv：cs.LG· Alessia Rossi, Felipe Campelo, Gabriel L. Oliveira·· 4 小时前
CAP-SAM：面向 SAM 的曲率感知自适应扰动方法
One Size Does Not Fit All: Curvature-Aware Adaptive Perturbation for SAM
AI 导读
针对 SAM 固定扰动半径 ρ 忽略局部曲率的问题，研究者提出曲率感知自适应扰动方法 CAP-SAM，依据 sharpness ratio 调节扰动半径，并证明其在非凸随机设定下保持与标准 SAM 相同的收敛速率。在 CIFAR-10、CIFAR-100 和 ImageNet 上，CAP-SAM 在多种网络架构中持续优于 SAM 及其主要变体。
正文
Abstract:Poor generalisation in deep neural networks is frequently associated with convergence to sharp minima. Sharpness-Aware Minimisation (SAM) improves generalisation of deep neural networks by minimising a combination of the training loss and the local sharpness of the loss landscape, biasing convergence towards flat minima. However, SAM relies on a fixed perturbation radius $\rho$ that ignores local curvature, leading to inefficient behaviour: the perturbation may be too small to escape poor basins defined by sharp local minima, or too large to allow convergence to solutions with good loss-sharpness tradeoff. We reinterpret $\rho$ as the intensity of the perturbation signal and propose Curvature-Aware Adaptive Perturbation for SAM (CAP-SAM), a geometry-aware method that modulates the perturbation radius based on a sharpness ratio, enabling a more effective search for robust minima on the loss landscape. We prove that CAP-SAM preserves the same convergence rate as standard SAM in non-convex stochastic settings. Extensive experiments indicate that our approach consistently outperforms SAM and its most prominent variants across diverse network architectures on CIFAR-10, CIFAR-100, and ImageNet.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.20415 [cs.LG] |
| (or arXiv:2610.20415v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20415 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Alessia Rossi [view email]
[v1]
Wed, 14 Oct 2026 16:12:56 UTC (349 KB)
来源：arXiv：cs.LG · arxiv.org