arXiv：cs.CL· Haeyong Kang, Chang D. Yoo·· 3 小时前
Draft-Guided Eviction：免训练 KV-Cache 压缩该何时驱逐而非保留什么
When to Evict, Not What to Keep: Draft-Guided Eviction for Training-Free KV-Cache Compression
AI 导读
研究者提出免训练 KV-Cache 压缩方法 Draft-Guided Eviction（DGE），将驱逐从 prefill 结束推迟到用完整缓存生成前 k=2 个答案 token 之后，仅多一步解码。
正文
Abstract:Training-free KV-cache compression methods such as SnapKV, H2O, and PyramidKV evict tokens at the end of prefill, aiming to preserve the attention mass that future queries are expected to use -optimizing what to keep. We show that this objective fails in two distinct ways. (1) Compensation: restoring the evicted attention mass can recover the attention-level target without recovering task quality. (2) Selection: covering more of the true decode-query mass can hurt quality when the recovered mass is fragmented rather than concentrated in coherent spans. These failures share a common cause: eviction occurs before the queries that determine the answer trajectory exist. We propose Draft-Guided Eviction (DGE), which defers eviction until after drafting the first k=2 answer tokens using the full cache - just one decode step beyond prefill. Because the draft is generated from the answer's own prefix, no cache entries are discarded before this trajectory signal becomes available. The per-head cache budget remains unchanged, and DGE can be applied directly to SnapKV, PyramidKV, H2O, and StreamingLLM without modifying their eviction scores. Unlike extra-pass methods, DGE changes when eviction occurs rather than what cache entries are selected. Extensive experiments demonstrate that DGE outperforms prior methods at every evaluated budget on five of six instruct-tuned backbones, achieving 44.2 on LongBench, nearly matching FullKV at 44.3. The timing-only control DGE-W achieves the same score, demonstrating that the gain comes from when eviction occurs rather than what is selected - an effect we term trajectory anchoring.
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.33334 [cs.LG] |
| (or arXiv:2609.33334v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.33334 arXiv-issued DOI via DataCite |
Submission history
From: Haeyong Kang [view email]
[v1]
Sun, 27 Sep 2026 08:04:59 UTC (335 KB)
[v2]
Thu, 8 Oct 2026 12:05:58 UTC (338 KB)
来源：arXiv：cs.CL · arxiv.org