arXiv：cs.AI· Vatsal, Soumil Malhotra, Priyanshu Chahal, Sanjeev Kumar·· 20 小时前
Prune-and-Merge ViT：无需训练的 Vision Transformer token 压缩
Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression
AI 导读
研究者提出 Prune-and-Merge ViT，一个面向现成模型的免训练框架，结合基于 class attention 的剪枝与基于相似度的合并来压缩 token 冗余，并引入 RAD 与 PFR 两项指标评估鲁棒性。在预训练的 ViT 和 DeiT 上，该方法在保持鲁棒性的同时计算量降低，准确率优于现有 token 减少方法，分析显示混合 token 压缩未引入类幻觉行为。
正文
Abstract:Vision Transformers achieve strong performance but incur high computational costs from quadratic token interactions. Training-free token reduction methods improve efficiency, but their robustness remains underexplored. We propose Prune-and Merge ViT, a training-free framework for off-the-shelf models. It combines class attention-based pruning with similarity-based merging to reduce redundancy while preserving semantic information. To evaluate robustness we introduce two metrics. Relative Accuracy Drop (RAD) measures relative accuracy degradation under transformations, and Prediction Flip Rate (PFR) quantifies prediction consistency. On pretrained ViT and DeiT, Prune-and-Merge reduces computation competitively while maintaining robustness and better accuracy than existing token reduction methods. Our analysis suggests that hybrid token reduction does not introduce hallucination-like behavior and preserves prediction stability for common image transformations.
| Comments: | Accepted at the GlobalSouthAI Workshop at NeurIPS 2026; 12 pages |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18039 [cs.CV] |
| (or arXiv:2610.18039v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18039 arXiv-issued DOI via DataCite |
Submission history
From: Vatsal Singhal [view email]
[v1]
Tue, 13 Oct 2026 09:29:25 UTC (22 KB)
[v2]
Wed, 14 Oct 2026 04:42:32 UTC (22 KB)
来源：arXiv：cs.AI · arxiv.org