arXiv：cs.LG· Tao Yu, Zhongtian Luo, Yuchen Long, Yiheng Lu, Yuxuan Zhou, Yanpei Gong, Siyi Cao, Yuqing Wu, Haopeng Jin, Zhenghao Zhang, Yujia Yang, Jingmin Zhu, Yunyun Hou, Xinming Wang, Hongzhu Yi, Jiachen Liu, Sirui Wang, Nan Duan, Yan Huang, Liang Wang, Tianjin Huang, Yanyan Zou·· 4 小时前
RSE-Bench：普通非对抗噪声如何损害智能体技能自进化
When Non-Adversarial Noise Hurts Learning: Benchmarking Skill Self-Evolution in Agents
AI 导读
大语言模型智能体通过将经验蒸馏为可复用技能来提升能力，但经验中的错误会固化进规则并影响后续任务。研究者提出 RSE-Bench，覆盖四个领域、六个基准，通过扰动指令、环境证据、执行记录和结果-记录配对来评测技能自进化在普通非对抗噪声下的鲁棒性，并配套 RSEClaw 框架实现一致的噪声注入与过程诊断。在三种方法和两种骨干模型上，带噪学习仅保留干净自进化所获总收益的 25.5%–42.1%。
正文
Authors:Tao Yu, Zhongtian Luo, Yuchen Long, Yiheng Lu, Yuxuan Zhou, Yanpei Gong, Siyi Cao, Yuqing Wu, Haopeng Jin, Zhenghao Zhang, Yujia Yang, Jingmin Zhu, Yunyun Hou, Xinming Wang, Hongzhu Yi, Jiachen Liu, Sirui Wang, Nan Duan, Yan Huang, Liang Wang, Tianjin Huang, Yanyan Zou
Abstract:Large language model agents increasingly improve by distilling experience into reusable skills, but errors in that experience can persist in learned rules and affect later tasks. Existing evaluations largely examine clean learning or perturbations encountered after skills are frozen, leaving robustness during self-evolution insufficiently understood. We introduce RSE-Bench, to our knowledge the first benchmark for evaluating skill self-evolution under ordinary, non-adversarial noise. It spans four domains and six benchmarks, with controlled perturbations to instructions, environmental evidence, execution records, and outcome--record pairings that preserve the underlying tasks and native scoring rules. The protocol separates noise during learning from input-side noise at evaluation, testing frozen skill stores on held-out tasks. We also introduce \textsc{RSEClaw}, a method-agnostic framework supporting consistent noise injection and process diagnostics of actual exposure and skill updates. Across three methods and two backbones, noisy learning retains only $25.5\%$--$42.1\%$ of the aggregate gain achieved by clean self-evolution, depending on the noise type. Perturbations confined to execution records and outcome--record pairings cause average losses comparable to input-side noise, despite leaving the original executions and evaluations intact. Diagnostics further identify differences in candidate skill text that coexist with small changes in mean held-out performance.
| Comments: | 35 pages, 2 figures |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.19910 [cs.LG] |
| (or arXiv:2610.19910v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19910 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tao Yu [view email]
[v1]
Wed, 14 Oct 2026 10:03:53 UTC (708 KB)
来源：arXiv：cs.LG · arxiv.org