arXiv：cs.CL· Hanzuo Liu, Xuan Qi, Chunyu Liu, Haotian Zhong, Yulong Wang, Key, Rayying, Alex Lamb, Mingyu Gao·· 3 小时前AI 评分42
CoMem：通过持久中间残差在多次查询间复用 Transformer 深度
CoMem: Reusing Transformer Depth across Queries with Persistent Intermediate Residuals
AI 导读
CoMem 将 Transformer 的分割深度 j 变为可复用的显式上下文轴，每个 token 写一份深度 j 残差，选中有限 chunk 后只重跑 [j:L) 层。
正文
Abstract:Repeated queries over shared documents repeatedly execute the same lower transformer layers. We introduce CoMem, which makes split depth j an explicit reusable-context axis: write one depth-j residual per token, select a bounded chunk set, and resume only layers [j:L). Among document-reuse systems we are aware of, CoMem jointly makes split depth a tunable serving axis and isolates it with a matched j=0 endpoint. On Qwen3-8B, j=12 reduces selected-pack Read from 931.9 to 664.4 ms (1.403x), with a 3.12-point RULER cost (95% CI [2.36, 3.93]); a continuous-prefix oracle recovers the full gap. The resulting depth axis quantifies a quality-latency-storage trade-off; a separate same-adapter, Write-inclusive pipeline is 2.74x faster. Equal-latency raw replay leads by 11.56 points with BM25, directly measuring an applicability boundary of prepaid depth rather than hiding it. CoMem stores 8 KiB/token versus 144 KiB/token for a protocol-aligned same-Qwen3 CacheBlend-style diagnostic; the cohorts and adaptation budgets are not matched. A context-position factorization identifies missing lower-layer document context as the dominant tested multikey error, and a 32-token overlap raises 92.5 to 98.5 without increasing persistent bytes or per-query Read. CoMem opens transformer depth as a measurable, tunable dimension for repeated-query long-context serving.
| Comments: | 32 pages, 2 figures. Published at the COLM 2026 Workshop on Efficient Reasoning |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.28263 [cs.CL] |
| (or arXiv:2607.28263v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.28263 arXiv-issued DOI via DataCite |
Submission history
From: Liu Hanzuo [view email]
[v1]
Thu, 30 Jul 2026 14:19:11 UTC (287 KB)
[v2]
Wed, 7 Oct 2026 13:49:53 UTC (268 KB)
来源：arXiv：cs.CL · arxiv.org