arXiv：cs.LG· Vakul Raghavan, Utkarsh Grover, Mingyang Mao, Chengwen Wang, Xiaomin Lin·· 13 小时前
Double-Take：在双向记忆中提升长上下文容量、绑定与选择性
Double-Take: Improving Capacity, Binding, and Selectivity In Long-Context Within Bidirectional Memory
AI 导读
针对大块测试时训练（TTT）中单步写入干扰限制容量、自关联写入无法形成跨位置绑定的两个缺陷，研究者提出 Double-Take（DT），用多步非固定更新、跨位置混合和双向变化敏感写入修复。
正文
Abstract:Processing a long sequence does not guarantee that distant information remains usable. As context grows, a fixed-size memory must retain many associations, connect information across positions, and preserve relevant structure without being overwhelmed by everything it observes. These are distinct requirements: a memory may store more information without representing the relationships needed for retrieval, and it may transport those relationships without preserving the ones that matter. Thus, the challenge is not simply extending context length, but constructing a memory that remains usable as context grows. We study this problem in large-chunk test-time training (TTT) and identify two basic failures: single-step writes accumulate interference, limiting capacity, while self-associative writes cannot form cross-position bindings. We introduce Double-Take (DT), which repairs these failures with multi-step unpinned updates and cross-position mixing, and adds bidirectional change-sensitive writes for selective retention. Capacity alone leaves end-to-end recall at the $0.10$ chance floor; adding binding raises it to $0.92$. Trained only on $32\,\mathrm{kb}$ genomic contexts, DT reaches $0.927$ AUROC at $131\,\mathrm{kb}$ and $0.856$ near $1\,\mathrm{Mb}$, while transferring zero-shot from human to mouse. Together, these results identify capacity, binding, and selectivity as three requirements for usable long-range memory.
| Comments: | 23 pages, 4 figures, 15 tables |
| Subjects: | Machine Learning (cs.LG); Genomics (q-bio.GN); Quantitative Methods (q-bio.QM) |
| Cite as: | arXiv:2610.13073 [cs.LG] |
| (or arXiv:2610.13073v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13073 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vakul Raghavan [view email]
[v1]
Fri, 9 Oct 2026 06:31:57 UTC (325 KB)
来源：arXiv：cs.LG · arxiv.org