arXiv：cs.LG（机器学习，全量分类）· Zonghuan Xu, Xingjun Ma·· 1 天前AI 评分34
从顺序到分布：持续学习中遗忘的精确算子框架
From Order to Distribution: An Exact Operator Framework for Forgetting in Continual Learning
AI 导读
一项持续学习理论研究将遗忘分析从任务顺序转向任务分布，在共享解、i.i.d. 任务采样与顺序精确拟合的线性设定下，推导出用任务分布直接表达历史遗忘的精确算子恒等式。基于该恒等式，研究给出有限维下任意固定任务分布期望历史遗忘的指数衰减保证，并将衰减与分布对可观测方向的覆盖度关联。针对单个已学任务，后续任务无需精确回访即可协同支持恢复，研究还给出恢复时间下界并构造了达到其逆覆盖度缩放的任务分布。
正文
Abstract:A central challenge in continual learning is forgetting: the loss of performance on previously learned tasks after learning new ones. Prior theory has analyzed forgetting under random orderings of fixed task collections in overparameterized linear regression. We shift the focus from task order to task distribution, asking how its structure determines forgetting. In the linear setting with a shared solution, i.i.d. task sampling, and sequential exact fitting, we derive an exact operator identity expressing historical forgetting directly in terms of the task distribution. Building on this identity, we establish an exponential decay guarantee for expected historical forgetting under every fixed task distribution in finite dimensions, characterize its asymptotic behavior, and relate decay to the distribution's coverage of observable directions. For an individual learned task, we show that subsequent tasks can collectively support recovery without exact revisits. We derive a lower bound on recovery time and construct a task distribution attaining its inverse-coverage scaling.
| Comments: | 34 pages, 3 figures. Revised analysis and proofs; new fixed-operator comparisons and recovery results |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.13460 [cs.LG] |
| (or arXiv:2604.13460v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.13460 arXiv-issued DOI via DataCite |
Submission history
From: Zonghuan Xu [view email]
[v1]
Wed, 15 Apr 2026 04:29:00 UTC (743 KB)
[v2]
Thu, 1 Oct 2026 02:37:09 UTC (4,221 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org