arXiv：cs.AI· Tianyu Jin, Liuyu Xiang, Chaowen Yang, Xingyang Chang, Pei Pei Li, Zhaofeng He·· 16 小时前
FARL：全自动强化学习接口与求解器设计流水线
FARL: A Fully Automated Pipeline of RL Interface and Solver Design
AI 导读
FARL 是一个全自动强化学习设计框架，联合搜索观测与奖励程序、学习算法及训练配置，并通过训练统计与行为诊断失败、提出并验证改进方案。在 Adroit Hand Relocate 任务上，FARL 以每策略 5M 训练步达到 97.1% 成功率，而 Eureka 和 LiMEN 为 0%、原生 PPO 为 1.0%。该框架在导航、运动和操作等挑战性任务上优于所评估的自动化基线。
正文
Abstract:Existing reinforcement learning (RL) largely focuses on optimizing learning algorithms within a pre-defined task formulation, while applying RL to new tasks still relies heavily on human expertise to design the learning pipeline: what an agent observes, how it is rewarded, which algorithm it uses, and how training is configured. Automating these choices is challenging because they are tightly coupled. We introduce FARL (Fully Automated Reinforcement Learning), a framework that automates RL design by jointly searching observation and reward programs, learning algorithms, and training settings. FARL uses training statistics and observed behavior to diagnose failures, propose changes, and test whether they improve learning. Its search preserves diverse candidates and the controls needed for these tests, using a staged budget to screen designs before investing in longer training. Experimental memory records each finding together with its conditions, so an unsuccessful component can be reconsidered in a different combination. Across expert-withheld and expert-available settings, FARL outperforms the evaluated automated baselines on challenging navigation, locomotion, and manipulation tasks. On Adroit Hand Relocate, FARL achieves 97.1% success at 5M training steps per policy, compared with 0% for Eureka and LiMEN and 1.0% for native Proximal Policy Optimization (PPO) under the same policy training budget.
| Comments: | 43 pages, including references and appendices. Code and recorded experimental results are included as ancillary files |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.15691 [cs.AI] |
| (or arXiv:2610.15691v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15691 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tianyu Jin [view email]
[v1]
Sun, 11 Oct 2026 15:22:18 UTC (3,166 KB)
来源：arXiv：cs.AI · arxiv.org