HuggingFace Daily Papers·· 1 天前
从 VLM 规划器到机器人智能体：面向历史依赖操作的自进化 harness
From VLM Planners to Robotic Agents: Self-Evolving Harnesses for History-Dependent Manipulation
AI 导读
研究提出 SERA（Self-Evolving Robotic Agent），把高层 VLM 组织成可编辑 harness 来维护任务上下文，底层由固定 VLA 执行器控制，并借助环境 rollout 提出和评估 harness 修订实现自进化。在 RoboMME 上，SERA 通过自主选择取得 51.38% 的整体成功率，相比不含自进化的 44.38% 有所提升。
正文 · 原文
Authors:Tingwen Du, Haoyu Guo, Hanxue Zhang, Xuanxu Wang, Junting Dong, Chao Gao, Mingfei Han, Tao Lu, Mulin Yu, Chunhua Shen, Xiaojun Chang, Zhihui Li
Abstract:Vision-language-action (VLA) models achieve strong performance across embodied manipulation tasks by unifying visual understanding, language conditioning, and action generation. However, long-horizon tasks can involve non-Markovian dependencies: successful decisions may require retaining task progress and past interaction outcomes, while partial observability and execution errors compound over time. Existing work has begun to address these challenges by augmenting foundation-model policies with memory and by introducing hierarchical architectures that separate high-level planning from low-level execution. We organize the high-level VLM as a robotic agent through an editable harness to maintain task context and improve planning through execution experience. We instantiate this approach as SERA (Self-Evolving Robotic Agent), with a fixed VLA executor providing low-level control. The harness maintains and updates task context using observations and execution feedback to guide subsequent planning. Self-evolution uses environment rollouts to propose and evaluate harness revisions, with the selected configuration fixed before held-out testing. On RoboMME, SERA achieves 51.38% overall success with autonomous selection, compared with 44.38% without self-evolution. Our exploration suggests that agentic planning is a promising direction for history-dependent robotic manipulation.
| Subjects: | Robotics (cs.RO) |
| Cite as: | arXiv:2610.29560 [cs.RO] |
| (or arXiv:2610.29560v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29560 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tingwen Du [view email]
[v1]
Wed, 21 Oct 2026 10:49:49 UTC (6,654 KB)
来源：HuggingFace Daily Papers · arxiv.org