Varest：通过预 LLM 语义可变分辨率 Token 化与 LLM 内相关性剪枝实现图像 Token 压缩

Just Respect the Semantics: Image-Token Compression via Pre-LLM Semantic Variable-Resolution Tokenization and In-LLM Relevance Pruning

AI 导读

Varest 提出基于语义可替代性的图像 Token 压缩方法，通过目标与文本定位器引导可变分辨率合并，并用学习到的 [REL] 寄存器在解码器早期处理后剪枝视觉 Token。

正文

View PDF HTML (experimental)

Abstract:Image-token compression must reduce computation without hurting the semantics. We introduce Varest, guided by semantic fungibility: tokens within a coherent semantic region can be summarized coarsely, while object boundaries and literal text require finer distinctions. Object and text localizers guide variable-resolution merging, preserving spatial coverage and finer detail at boundaries and text. Learned [REL] registers then prune visual tokens after early decoder processing. Training across budgets enables one checkpoint per backbone. Across three backbones and 13 benchmarks, Varest outperforms 15 baselines in mean performance retention at every evaluated budget, retaining 86.10-92.33% of full-token performance with only 5.56% of the original layer-averaged image-token count.

Subjects:	Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as:	arXiv:2610.18690 [cs.CV]
	(or arXiv:2610.18690v1 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2610.18690 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Tianhao Fu [view email]
[v1] Tue, 13 Oct 2026 16:42:32 UTC (4,734 KB)

来源：arXiv：cs.AI · arxiv.org