arXiv：cs.CL· Haoyu Huang, Zhongwei Xie, Jiaxin Bai, Yisen Gao, Hong Ting Tsang, Wuganjing Song, Huihao Jing, Yufei Li, Yangqiu Song·· 6 小时前AI 评分34
面向大语言模型的参数内记忆增强综述
Towards In-Parameter Memory Augmentation for Large Language Models
AI 导读
一篇综述聚焦于在部署阶段为 LLM 增强「参数内记忆」的方法，即把承载记忆的参数对象在推理时插入前向传播，无论其获取于部署前还是部署中。文章用参数放置位置（Embedding、Attention、FFN 层或 Hybrid）与参数获取时间（在线/离线）两条正交轴梳理该领域，并讨论了干扰、安全、与 ICL 协同设计及递归自我改进等开放方向。
正文
Abstract:Recently Large Language Models (LLMs) and LLM-based agents increasingly need to incorporate knowledge acquired after pretraining, e.g., domain facts, user preferences, documents, and interaction experience. In-context learning (ICL) and ICL-based agent harness remain flexible, but they consume context capacity and incur repeated discretized encoding cost that grows with context length. \textbf{In-parameter memory} offers a complementary substrate: reusable memory information is represented in model parameters, adapters, or other parameter-like objects that are composed into the forward pass at inference time. This survey focuses on methods that augment LLMs with such parametric memory at deployment: a memory-bearing parameter object is plugged into the forward pass during inference, whether it is acquired before or during deployment. We organize the landscape with two orthogonal axes: \textbf{Parameter Placement}, which includes Embedding, Attention, FFN layers, or Hybrid when two or more layers are used; and \textbf{Parameter Acquisition Time}, which distinguishes methods whose memory object is acquired during deployment (online) from those acquired before it (offline). We clarify boundaries, conduct comparisons, and discuss open directions in interference, safety, co-design with ICL, and recursive self-improvement.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.08630 [cs.CL] |
| (or arXiv:2610.08630v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.08630 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Haoyu Huang [view email]
[v1]
Tue, 6 Oct 2026 16:27:47 UTC (3,403 KB)
来源：arXiv：cs.CL · arxiv.org