arXiv：cs.LG· Carlos Plou, Ana C. Murillo, Ruben Martinez-Cantin·· 10 小时前
基于贝叶斯模型的强化学习在机器人操作中的主动探索
Active Exploration in Bayesian Model-based Reinforcement Learning for Robot Manipulation
AI 导读
研究将贝叶斯深度学习用于机器人操作的基于模型强化学习，通过改善动态模型校准质量，并在初步探索阶段主动估计每次状态转移的信息增益、将其作为探索奖励来实现主动学习，从而提升数据效率。实验在 CoppeliaSim 机械臂仿真器上完成端到端任务，结果显示该方法能以更低的机器人执行需求达到与相关替代方案相当的质量。
正文
Abstract:Efficiently tackling multiple tasks within complex environments, such as those found in robot manipulation, remains an ongoing challenge in robotics and an opportunity for data-driven solutions such as reinforcement learning (RL). The high cost of data gathering in robotics encourages data-efficient approaches such as model-based RL, which builds a dynamic model of the robot to conduct most policy learning on cheaper simulations, while enabling data reuse and transfer between tasks. In this work, we employ Bayesian deep learning (BDL) with a double purpose. First, we increase the model quality by improving its calibration. Second, we improve data efficiency by performing active learning of the dynamic model during a preliminary exploration phase, actively estimating the information gain of each transition and using it as the exploration reward. We compare several BDL methods in a realistic robot manipulation setup. Our experiments show that our approach matches the quality of relevant alternatives with much lower robot execution requirements and, unlike previous studies validated only on standard RL environments, we tackle robotic arm end-tasks on a complete manipulator simulator such as \texttt{CoppeliaSim}. Code is available at this https URL.
| Subjects: | Robotics (cs.RO); Machine Learning (cs.LG) |
| Cite as: | arXiv:2404.01867 [cs.RO] |
| (or arXiv:2404.01867v3 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2404.01867 arXiv-issued DOI via DataCite |
Submission history
From: Carlos Plou [view email]
[v1]
Tue, 2 Apr 2024 11:44:37 UTC (30,453 KB)
[v2]
Fri, 9 Oct 2026 18:43:58 UTC (9,014 KB)
[v3]
Wed, 14 Oct 2026 08:48:10 UTC (9,014 KB)
来源：arXiv：cs.LG · arxiv.org