MemoryAthena：在隐式与生成记忆上做自适应路由

View PDF HTML (experimental)

Abstract:Learned-memory methods store information in an explicit table and consume it through a separate reader, allowing addressing, storage, and reading to be modified independently. We study whether useful memory can also be generated rather than only retrieved. MemoryAthena uses three pathways: direct Engram retrieval (E), generation from retrieved Engram cues (GE), and generation from causal backbone states without consulting the memory table (GH). Generated memory is conditionally useful: it can complement E in one context but interfere with it in another. MemoryAthena therefore treats E as an anchor and learns when a generated representation should intervene. With the backbone, memory, generators, and readers frozen, a lightweight causal routing head is trained from counterfactual future-token likelihood advantages of GE and GH relative to E. At inference time, an admitted candidate modifies the E residual through bounded interpolation, while rejection recovers the direct pathway exactly. On question answering, MemoryAthena raises the five-task average from 37.65 to 39.28 over the direct pathway of the same checkpoint, while the six-task general-NLP average increases from 76.73 to 79.13. The complete memory-side system contains approximately 201M parameters, excluding the frozen backbone. Further analyses show complementary strengths among E, GE, and GH across tasks and inputs. These results support generated memory as a selective correction to direct retrieval and highlight routing when, which, and how strongly to intervene as the central challenge.

Subjects:	Computation and Language (cs.CL)
Cite as:	arXiv:2609.25853 [cs.CL]
	(or arXiv:2609.25853v3 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2609.25853 arXiv-issued DOI via DataCite

Submission history

From: Mingyuan Li [view email]
[v1] Tue, 22 Sep 2026 08:17:43 UTC (2,266 KB)
[v2] Sat, 26 Sep 2026 19:52:03 UTC (2,395 KB)
[v3] Wed, 14 Oct 2026 08:23:38 UTC (2,397 KB)