arXiv：cs.AI· Lute Lillo, Nick Cheney·· 20 小时前
TeLAPA：在持续强化学习中用策略档案库保留可塑性
Beyond Single-Model Optimization: Preserving Plasticity in Continual Reinforcement Learning
AI 导读
研究提出持续强化学习框架 TeLAPA（Transfer-Enabled Latent-Aligned Policy Archives），通过按任务组织行为多样的策略邻域并维护共享隐空间，让存档策略在非平稳漂移下仍可比较与复用。
正文
Abstract:Continual reinforcement learning must balance retention with adaptation, yet many methods still rely on \emph{single-model preservation}, committing to one evolving policy as the main reusable solution across tasks. Even when a previously successful policy is retained, it may no longer provide a reliable starting point for rapid adaptation after interference, reflecting a form of \emph{loss of plasticity} that single-policy preservation cannot address. Inspired by quality-diversity methods, we introduce \textsc{TeLAPA} (Transfer-Enabled Latent-Aligned Policy Archives), a continual RL framework that organizes behaviorally diverse policy neighborhoods into per-task archives and maintains a shared latent space so that archived policies remain comparable and reusable under non-stationary drift. This perspective shifts continual RL from retaining isolated solutions to maintaining \emph{skill-aligned neighborhoods} with competent and behaviorally related policies that support future relearning. In our MiniGrid CL setting, \textsc{TeLAPA} learns more tasks successfully, recovers competence faster on revisited tasks after interference, and retains higher performance across a sequence of tasks. Our analyses show that source-optimal policies are often not transfer-optimal, even within a local competent neighborhood, and that effective reuse depends on retaining and selecting among multiple nearby alternatives rather than collapsing them to one representative. Together, these results reframe continual RL around reusable and competent policy neighborhoods, providing a route beyond single-model preservation toward more plastic lifelong agents.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE) |
| Cite as: | arXiv:2604.15414 [cs.LG] |
| (or arXiv:2604.15414v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2604.15414 arXiv-issued DOI via DataCite |
Submission history
From: Lute Lillo [view email]
[v1]
Thu, 16 Apr 2026 17:06:54 UTC (4,491 KB)
[v2]
Tue, 9 Jun 2026 00:58:37 UTC (6,378 KB)
[v3]
Tue, 13 Oct 2026 17:46:41 UTC (6,378 KB)
来源：arXiv：cs.AI · arxiv.org