arXiv：cs.CL· Hanchen Xia, Baoyou Chen, Yutang Ge, Naihao Deng, Senqiao Yang, Zilong Dong, Weihao Yuan, Siyu Zhu·· 3 小时前
REMORY：为上下文压缩学习残差记忆
REMORY: Learning Residual Memory for Context Compaction
AI 导读
REMORY 是一种神经记忆网络，在文本摘要之外生成一段有界的软记忆 token，让冻结的 LLM 更接近使用完整历史时的输出。在 SummHay 上，REMORY 仅用 5.2% 的输入位置就提升了来源归因，并接近全上下文联合得分。
正文
Abstract:Long-horizon agents compact their history to continue within a finite context window, but a textual summary alone may not support every subsequent decision. We introduce REMORY, a neural memory network that supplements the summary with a bounded sequence of soft memory tokens. Given the history and summary, the network learns to generate tokens that help a frozen LLM approximate the continuation it would produce with the full history. The tokens are conditioned on the summary and appended after it, forming an analogue of a residual connection along the sequence dimension. On SummHay, REMORY improves source attribution at nearly unchanged insight coverage and approaches the full-context joint score using only 5.2% of the input positions. Across long-horizon agent benchmarks, Qwen3.8-27B and GLM-5.3-Flash show consistent gains with residual memory. Both models also exhibit substantially fewer repeated tool outputs and tool errors on BrowseComp and Terminal-Bench 2.1.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.11287 [cs.CL] |
| (or arXiv:2610.11287v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11287 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hanchen Xia [view email]
[v1]
Thu, 8 Oct 2026 05:50:50 UTC (1,645 KB)
来源：arXiv：cs.CL · arxiv.org