arXiv：cs.LG· Haoru Li, Jinmei Liu, Zhiyong Wang, Xiaoming Li, Zhenhong Sun, Daoyi Dong, Chunlin Chen, Zhi Wang·· 4 小时前AI 评分38
DRIVE：面向 VLA 泛化的多样性驱动 RL 微调
Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization
AI 导读
研究者提出 DRIVE（Diversity-driven RL fIne-tuning for VLA gEneralization），将成功行为多样性转化为显式 RL 目标，通过对齐比较同一任务条件下的轨迹生成成功率条件的内在奖励。
正文
Abstract:Reinforcement learning (RL) fine-tuning improves vision-language-action (VLA) policies through closed-loop experience, yet generalization beyond the fine-tuning distribution remains limited. Our analysis reveals a selective reshaping of exploration: RL contracts behavior globally, yet diversifies successful trajectories, elicits success with fewer rollouts, and covers more of the latent task-valid solution space than supervised fine-tuning. Broader successful-mode coverage may provide alternative strategies under distribution shifts. Inspired by this, we introduce DRIVE (Diversity-driven RL fIne-tuning for VLA gEneralization), which turns successful-behavior diversity into an explicit RL objective. DRIVE groups rollouts under matched task conditions, compares their trajectories with temporal alignment, and derives a success-conditioned intrinsic reward from relative behavioral diversity. This design encourages broader coverage of feasible solutions without rewarding diverse failures or superficial timing differences. Across LIBERO-Plus, ManiSkill3, and RoboTwin 2.0, DRIVE improves the average out-of-domain (OOD) performance over vanilla RL fine-tuning by 5.3 points on $\pi_0$ and 2.0 points on $\pi_{0.5}$. On a dual-arm AgileX PiPER-X platform, DRIVE further increases average OOD success from 64.1% to 73.3% (+9.2 points), demonstrating gains that persist under physical deployment.
| Subjects: | Robotics (cs.RO); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.09943 [cs.RO] |
| (or arXiv:2610.09943v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09943 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Haoru Li [view email]
[v1]
Wed, 7 Oct 2026 12:26:21 UTC (9,826 KB)
来源：arXiv：cs.LG · arxiv.org