arXiv：cs.LG（机器学习，全量分类）· Samuel Larson·· 14 小时前AI 评分34
DeltaNet 固定矩阵状态实现常量内存召回：32 KiB 状态达 99.95% 准确率
Constant-Memory Recall: Learned Associations in a Fixed Matrix State
AI 导读
一项研究用带固定 token 键偏置的小型 DeltaNet 变体，在 32 KiB 循环矩阵状态下记忆每序列 32 对新键值对，三个训练种子平均准确率达 99.95%。填充内容将查询前上下文延长至 1,798 tokens 时召回仍近乎完美，而将首个记忆块置零会消除该召回。参数匹配的向量与 Transformer 基线仍接近随机水平，导致无法进行内存效率对比。
正文
Abstract:Fixed-size recurrent memory limits storage growth during inference, but successful recall depends on the task and training. We study a small DeltaNet variant with fixed token-specific key biases, trained to remember 32 new key-value pairings per sequence. With 32 KiB of recurrent matrix state, it achieves 99.95% mean accuracy across three training seeds when choosing among the sequence's values. Recall remains near perfect when filler extends the pre-query context to 1,798 tokens without adding pairings. Zeroing the first memory block removes this recall. An exploratory 48-pair test remains near chance after one quarter of the primary training budget and does not locate a capacity limit. Parameter-matched vector and Transformer baselines remain near chance, including the Transformer after additional training searches. This unresolved baseline failure prevents a memory-efficiency comparison.
| Comments: | 8 pages, 3 figures |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.00232 [cs.LG] |
| (or arXiv:2610.00232v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00232 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Samuel Larson [view email]
[v1]
Wed, 23 Sep 2026 00:17:36 UTC (311 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org