arXiv：cs.LG（机器学习，全量分类）· Sima Hashemi, Daniel Durstewitz, Georgia Koppe·· 5 小时前AI 评分36
基于可回收单元门控的循环神经网络动态系统持续学习
Continual Learning of Dynamical Systems in Recurrent Neural Networks through Recyclable Unit Gating
AI 导读
研究者提出 Continually-Recyclable Unit-Gating（CRUG），在 Almost-Linear RNN（AL-RNN）上实现零遗忘的持续动态系统重建（cDSR）。该方法用基于 L0 惩罚的可微门控选择任务专属单元，并将未使用单元回收给后续任务，在重建质量与容量权衡上优于参数正则化、回放和参数隔离等持续学习方法，还能可靠学习非线性与混沌系统序列。
正文
Abstract:Dynamical Systems Reconstruction (DSR) aims to infer models from observed time series that reproduce a system's qualitative long-term behavior. Continual DSR (cDSR) requires learning new systems while preserving previously learned dynamics, yet even small parameter updates in recurrent models can qualitatively alter their behavior over long autonomous rollouts. We benchmark established continual learning (CL) methods spanning parameter regularization, replay, and parameter isolation on the fully trainable and interpretable Almost-Linear RNN (AL-RNN). Parameter isolation preserves earlier dynamics most effectively, but excessive task-specific allocations can rapidly exhaust a fixed-size network. We therefore introduce Continually-Recyclable Unit-Gating (CRUG), which conserves capacity through compact allocation and forward transfer. Differentiable gates trained with an $L_0$-based penalty select task-specific units, while unused units are recycled for subsequent tasks. Directed connections allow later tasks to reuse earlier representations without affecting the dynamics of previously committed units. CRUG achieves the strongest reconstruction--capacity trade-off among the tested methods with zero forgetting and reliably learns a heterogeneous sequence of nonlinear and chaotic systems. Furthermore, we show that forward transfer is more pronounced and useful when tasks share similar underlying dynamics. Lastly, we demonstrate that CRUG's advantages extend beyond autonomous cDSR to sequential cognitive tasks.
| Comments: | 34 pages (including Appendix), 15 Tables, 7 Figures |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2609.38356 [cs.LG] |
| (or arXiv:2609.38356v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.38356 arXiv-issued DOI via DataCite |
Submission history
From: Sima Hashemi [view email]
[v1]
Tue, 29 Sep 2026 18:18:32 UTC (4,179 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org