arXiv：cs.AI· Yuqian Fu, Kaiwen Jiang, Zijun Song, Jiawei Wang, Yuanheng Zhu, Dongbin Zhao·· 20 小时前
TSRL：基于触发机制的监督强化学习实现智能体持续训练
TSRL: Trigger-based Supervised Reinforcement Learning for Continual Agent Training
AI 导读
研究者提出 TSRL（Trigger-based Supervised Reinforcement Learning），以 RL 为优化主干、仅在策略失败的"hard states"上选择性激活 SFT，将监督限制在失败状态可导出累积参数更新上界，从而减少遗忘。
正文
Abstract:Large language model (LLM) agents excel at complex, multi-step tasks but suffer from catastrophic forgetting during continual learning--acquiring new skills often degrades previously mastered capabilities. Existing post-training paradigms face a fundamental trade-off: Supervised Fine-Tuning (SFT) is sample-efficient but prone to severe forgetting, while Reinforcement Learning (RL) preserves prior capabilities but converges slowly without expert guidance. To bridge this gap, we introduce Trigger-based Supervised Reinforcement Learning (TSRL), a framework that integrates the strengths of both paradigms through an adaptive, state-dependent expert supervision mechanism. TSRL employs RL as the optimization backbone, activating SFT selectively on "hard states" where the policy fails. Theoretically, we show that restricting supervision to failure states derives an upper bound on the cumulative parameter updates, thereby reducing forgetting compared to full SFT. Empirically, TSRL achieves consistent gains across three agent benchmarks (ALFWorld, WebShop, and Deep Search) at both 7B and 32B scales. In continual learning scenarios, our method retains up to 9.6 points more capability on prior tasks than SFT->RL baselines, while matching or exceeding target-task performance.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.17726 [cs.LG] |
| (or arXiv:2610.17726v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17726 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuqian Fu [view email]
[v1]
Tue, 13 Oct 2026 05:02:58 UTC (548 KB)
来源：arXiv：cs.AI · arxiv.org