arXiv：cs.AI· Himanshu Udupi, Calvin Ang, Sungyoon Kim, Xiaocong Yang, ChengXiang Zhai·· 13 小时前
脉冲神经网络（SNN）的凸优化：具备全局最优与近似保证
Convex Optimization of Spiking Neural Networks with Global and Approximation Guarantees
AI 导读
研究将脉冲神经网络的训练构建为等价的凸优化问题，证明一类阈值 RNN（含 leaky integrate-and-fire SNN）在权重衰减正则下零对偶间隙，且与凸程序最优值相同。
正文
Abstract:Spiking neural networks (SNNs) have been proposed as energy-efficient alternatives to deep neural networks. Their training is non-convex and involves a non-differentiable spike function, so existing methods offer no guarantee of reaching a global optimum. We address this issue by formulating SNN optimization as an equivalent convex optimization problem. Specifically, we show that the weight-decay-regularized training problem for a class of threshold RNNs, including leaky integrate-and-fire SNNs, has zero duality gap and the same optimal value as a convex program. Since the resulting convex programs can have exponentially many variables in the amount of data, we introduce a practical randomized surrogate and establish a high-probability guarantee on its approximation quality. Empirically, the best convex variant outperforms surrogate-gradient training on addition and first-last-XOR, matches its performance on sequential MNIST, and trails a reservoir baseline on base-3 and base-5 addition.
| Subjects: | Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.08022 [cs.NE] |
| (or arXiv:2605.08022v3 [cs.NE] for this version) | |
| https://doi.org/10.48550/arXiv.2605.08022 arXiv-issued DOI via DataCite |
Submission history
From: Xiaocong Yang [view email]
[v1]
Fri, 8 May 2026 17:10:08 UTC (2,655 KB)
[v2]
Fri, 26 Jun 2026 20:47:07 UTC (2,652 KB)
[v3]
Fri, 9 Oct 2026 03:25:50 UTC (3,270 KB)
来源：arXiv：cs.AI · arxiv.org