在 CIFAR-10 规范联合能量模型上对比 SGLD 与固定噪声 predictor-corrector 适配

View PDF HTML (experimental)

Abstract:Joint Energy-Based Models (JEM) unify classification and generation within a single network and support out-of-distribution (OOD) detection. Canonical JEM training relies on stochastic gradient Langevin dynamics (SGLD); a theoretically motivated alternative, the Predictor-Corrector (PC) sampler, has not previously undergone a systematic replication test on the canonical model. We reproduce canonical JEM on WideResNet-28-10 without normalisation layers on two independent runs and test a fixed-noise PC adaptation - with the degenerate annealed-noise predictor replaced by a deterministic gradient step - across three protocols: the adapted sampler replacing SGLD throughout the full training trajectories (115-132 epochs); cold-start generation (FID); and refinement-style multi-OOD detection (AUROC). The reconstruction reaches 92.88% test accuracy and buffer-FID 44.46 (canonical: 92.90% and 38.40). We document two failure modes: catastrophic late-training divergence with the signature of the canonical outlier-buffer mechanism (all four runs), and run-dependent SVHN OOD-discrimination dynamics. No consistent method-level advantage of the adaptation over SGLD is observed on any protocol: refinement AUROC differences stay below 0.007 across ten checkpoint-OOD pairs; seeded cold-start generation favours SGLD by about five FID points; on the training protocol a hierarchical seed-by-image bootstrap gives a 95% confidence interval on the macro-averaged AUROC difference that contains zero, while a seed-level equivalence test with two runs per method cannot establish formal equivalence. The training-protocol data are consistent both with equivalence and with a small directional effect. This outcome is consistent with theory: the guarantees of the annealed-noise PC framework do not transfer to the constant-noise regime of canonical JEM.

Comments:	23 pages, 4 figures. v3: author's accepted manuscript (text of the final revision), title in the journal's sentence case. Published in Discover Computing 29, 755 (2026). The Version of Record of this article is published in Discover Computing, and is available online at this https URL
Subjects:	Machine Learning (cs.LG); Machine Learning (stat.ML)
Cite as:	arXiv:2608.05025 [cs.LG]
	(or arXiv:2608.05025v3 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2608.05025 arXiv-issued DOI via DataCite
Journal reference:	Discover Computing 29, 755 (2026)
Related DOI:	https://doi.org/10.1007/s10791-026-10679-9 DOI(s) linking to related resources

Submission history

From: Dmytro Knopov [view email]
[v1] Wed, 5 Aug 2026 16:28:57 UTC (1,743 KB)
[v2] Tue, 11 Aug 2026 09:27:21 UTC (1,747 KB)
[v3] Wed, 14 Oct 2026 11:51:50 UTC (1,750 KB)