arXiv：cs.CL· Xiaobing Yu, Peijie Qiu, Jin Yang, Xuanzhao Dong, Weiwei Ma, Zhaoqi An, Xiaoqi Zhao, Xiaofeng Liu·· 3 小时前
LadderEdit：面向 LLM 终身编辑的内存高效编辑级残差压缩
LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs
AI 导读
LadderEdit 通过压缩每次编辑后获得的 LoRA 适配器，在 LLaMA-3-8B、Mistral-7B 和 Qwen2.5-7B 上以 5.2 倍更少内存追平精确 LoRA 存储。
正文
Abstract:Lifelong editing of LLMs requires storing thousands of edits after acquisition. A widely used family of approaches attaches one LoRA adapter per edit, which preserves behavior but grows linearly in storage. To address this challenge, we propose LadderEdit, a method that compresses each LoRA adapter after it is acquired. Each edit is first stored at low rank as a cheap sketch. We then check whether this sketch still satisfies the rewrite, generalization, and locality contract on probe prompts. Edits that pass keep the sketch; those that fail are promoted to a higher rank along a ladder until the contract is met. Because every edit retains some representation, coverage is maintained, and only hard edits consume more rank. Across ZsRE, CounterFact, and WikiBigEdit benchmarks on LLaMA-3-8B, Mistral-7B, and Qwen2.5-7B, LadderEdit tracks exact LoRA storage at 5.2x less memory and remains effective at 50,000 sequential edits.
| Comments: | EMNLP 2026 Main Conference Long Paper |
| Subjects: | Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Multimedia (cs.MM) |
| Cite as: | arXiv:2610.11160 [cs.CL] |
| (or arXiv:2610.11160v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11160 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiaofeng Liu [view email]
[v1]
Thu, 8 Oct 2026 03:20:35 UTC (531 KB)
来源：arXiv：cs.CL · arxiv.org