无反向传播的节能深度学习：前向算法的严格硬件验证基准测试研究

View PDF HTML (experimental)

Abstract:Backpropagation (BP) powers modern deep learning, but it must hold every activation in memory until a global backward sweep consumes it, and that sweep locks each layer until the one above has finished. Forward-only algorithms, from Hinton's Forward-Forward (FF) through Cascaded Forward (CaFo) to Mono-Forward (MF), replace the global gradient with local learning signals and are promoted as more efficient on the basis of FLOP counts rather than measurements. We test that claim on hardware. Across almost 1000 runs on four vision benchmarks (MNIST, Fashion-MNIST, CIFAR-10 and CIFAR-100), we read energy, power and memory from an NVIDIA A100 GPU and compare each algorithm against a tuned BP baseline. Removing the backward pass does not by itself save energy: FF costs 4.1x to 6.3x BP's energy and CaFo 3.0x to 7.1x. Mono-Forward is the exception. Given the same number of data passes as BP, it uses 12.5% to 19.9% less energy and 24.6% to 27.5% less memory on the 3x2000 multilayer perceptrons, the largest configurations tested, at lower power and near-equal wall-clock time; on CIFAR-100 its test accuracy is statistically equivalent to BP's. A pre-registered ablation identifies gradient detachment, not MF's auxiliary losses, as the dominant factor tested. Trained to convergence, MF exceeds BP's test accuracy on CIFAR-10 and CIFAR-100 by 3.90 and 5.04 percentage points, but needs 3.5x to 4.5x as many passes and 3.2x to 4.0x the energy. Faster convergence is therefore the remaining target for an end-to-end gain.

Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
MSC classes:	68T07
ACM classes:	I.2.0
Cite as:	arXiv:2511.01061 [cs.LG]
	(or arXiv:2511.01061v2 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2511.01061 arXiv-issued DOI via DataCite

Submission history

From: Przemysław Spyra [view email]
[v1] Sun, 2 Nov 2025 19:48:44 UTC (1,721 KB)
[v2] Mon, 12 Oct 2026 10:45:22 UTC (116 KB)