arXiv：cs.AI· Shuxin Liu, Qing Liu, Yi Du, Ou Wu·· 4 小时前AI 评分40
CORE：面向 KV Cache 的覆盖校准与逐出质量重分配
CORE: COverage CAlibration and Evicted-Mass REdistribution for KV Cache
AI 导读
CORE 通过覆盖校准与逐出质量重分配改进 KV Cache 压缩，在 90% 压缩率下于三个骨干模型上最高超过最强 RULER 基线 3.78 分。该方法将查询效用与对数行列式覆盖的离线分配蒸馏为轻量缓存感知索引器，推理时用同一校准分布同时驱动 Top-B 保留与潜在记忆写入，无需单独的写入权重预测器或在线对数行列式评估。LongBench 与重复逐出评测进一步验证了其有效性与解码效率。
正文
Abstract:Long-context decoding is increasingly constrained by key--value (KV) cache memory and bandwidth. Existing fixed-budget compression methods typically separate retention from compensation, while a retention ranking specifies neither discarded attention mass nor the direction of induced output error. We start from an exact factorization: eviction error equals evicted attention mass times the directional gap between the evicted centroid and retained output, highlighting the importance of set-level coverage in retention and mass-preserving memory writing. We introduce CORE COverage Calibration and Evicted-Mass REdistribution for KV Cache, which distills an offline allocation combining query utility and log-determinant coverage into a lightweight cache-aware indexer. At inference, one calibrated distribution drives both channels: its Top-$B$ ordering retains complementary KV states, while its excluded allocation mass and conditional weights parameterize latent-memory writes without a separate write-weight predictor or online log-determinant evaluation. Our analysis provides a four-term pre-compensation error certificate, characterizes non-additive coverage interactions, and establishes mass-independent write stability with hierarchical bounds through recurrent and query-adaptive normalization. Across three backbones, CORE exceeds the strongest RULER baseline by up to 3.78 points at 90\% compression; LongBench and repeated-eviction evaluations further demonstrate strong effectiveness and decoding efficiency.
| Subjects: | Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.02235 [cs.AR] |
| (or arXiv:2610.02235v1 [cs.AR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.02235 arXiv-issued DOI via DataCite |
Submission history
From: Shuxin Liu [view email]
[v1]
Mon, 28 Sep 2026 08:31:48 UTC (1,281 KB)
来源：arXiv：cs.AI · arxiv.org