快手 KSA V2 技术报告：Summary Token 兼作 KV 内存与稀疏索引

Authors:OneRec Team, Chenglong Chu, Guohong Mu, Guowang Zhang, Hao Peng, Hui Wang, Jian Liang, Jiangxia Cao, Jiao Ou, Jiaxin Deng, Lu Ren, Muhao Wei, Ruiming Tang, Tianle Xu, Wenwu Ou, Xingmei Wang, Zhixin Ling, Zhiyuan Liang, Ziqi Wang, Chaoyi Ma, Haixiao Gao, Kuo Cai, Renpu Liu, Ruochen Yang, Wenlong Xu, Xinqi Jin, Yan Sun, Yifan Wang, Yingzhi He, Yiyu Wang, Yusen Huo, Zhaojie Liu, Zihan Zhou

View PDF HTML (experimental)

Abstract:Summary-token architectures such as Kwai Summary Attention (KSA) compress distant context into learnable Summary token per chunk, and hybrid variants interleave these layers with full attention to retain exact access to the original text. In such hybrid KSA-Full architecture, the interleaving inserted summary tokens are served as a streaming flow role across different layers. Although effective, however, it also introduce some bottlenecks: (1) the different layers summary compression chunk must be same, therefore hard to reach a flexible KV Cache compression, e.g., different layer set different summary compression chunk window. (2) in the full-attention layers, the summary token introduced additional computation cost, but such layer ignore them and still pay quadratic prefilling, therefore unfriendly to the computation side. Motivated by them, we developed the KSA V2, which not only achieves a more flexible KV cache memory compression but also reduces the computation cost in Full attention layer. Specifically, we found that the summary tokens can serve two roles: the KV memory and sparse indexer.

Submission history

From: Jiangxia Cao [view email]
[v1] Tue, 13 Oct 2026 16:18:44 UTC (2,157 KB)