BaCP：面向极稀疏神经网络表征保持的骨干对比剪枝

BaCP: Backbone Contrastive Pruning for Preserving Representations in Extremely Sparse Neural Networks

AI 导读

针对极稀疏度下非结构化剪枝常出现的表征坍塌问题，研究者提出骨干对比剪枝（BaCP），通过将稀疏网络的嵌入空间与预训练、微调及历史快照模型对齐来施加正则。基于 CAP 框架的对比分解，该工作在多种剪枝准则下给出了严格的匹配预算刻画，并在 90 组设置中验证：在标准剪枝失效的极稀疏区间，BaCP 显著提升准确率，而在表征完好时接近基线。

正文

View PDF HTML (experimental)

Abstract:Unstructured pruning at extreme sparsity often suffers from representational collapse, causing sharp drops in accuracy. To address this, we study Backbone Contrastive Pruning (BaCP), which regularizes the sparse network's embedding space by aligning it with pretrained, fine-tuned, and historical snapshot models. Building on the contrastive decomposition of the CAP framework (Xu et al., 2022), we provide a rigorous matched-budget characterization of this approach across multiple pruning criteria. Evaluated across 90 settings, BaCP improves accuracy substantially in extreme sparsity regimes where standard pruning fails, and is close to baseline where representations remain intact.

Subjects:	Machine Learning (cs.LG)
Cite as:	arXiv:2610.02524 [cs.LG]
	(or arXiv:2610.02524v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2610.02524 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Muhammad Haseeb [view email]
[v1] Thu, 1 Oct 2026 21:53:48 UTC (501 KB)

来源：arXiv：cs.LG · arxiv.org