arXiv：cs.LG· Iok Tong Lei, QianZhi Li, Ying Jie Yap, Yujie Zhang, Rui Zhong, Haichao Gui, Xiaolong Liu, Zhidong Deng·· 2 天前AI 评分36
ACE：用零样本工作流推理实现具身操作的智能体控制
ACE: Agentic Control for Embodied Manipulation via Zero-shot Workflow Reasoning
AI 导读
ACE 是一种智能体操作框架，将高层语言智能体与可复用的掩码条件视觉运动策略组合，在无需完整演示的情况下实现任务级零样本组合。在两个物理多步桌面任务 Semantic Formula Assembly 和 Constraint Retrieval 上，每任务 20 次随机试验中 ACE 分别达到 70% 和 80% 成功率，而无持久上下文时为 55% 和 70%。
正文
Abstract:General-purpose manipulation requires both semantic reasoning over task constraints and reliable execution of contact-rich actions. We present ACE, an agentic manipulation harness that composes a high-level language agent with a reusable mask-conditioned visuomotor policy. Given an open-ended instruction, the agent solves semantic constraints, binds objects to destination roles, and decomposes the task into executable transfers represented by tracked pick-and-place masks. Execution feedback supports outcome assessment, re-grounding, and retry, while persistent object and task context preserves earlier associations when manipulation changes visible cues. We evaluate ACE on two physical multi-step tabletop tasks, Semantic Formula Assembly and Constraint Retrieval. The visuomotor policy is trained only on generic pick-and-place demonstrations and reused without complete demonstrations of either evaluation task, enabling task-level zero-shot composition. Across 20 randomized trials per task, ACE achieves 70% and 80% success, respectively, compared with 55% and 70% without persistent context. These results suggest that an agentic harness can extend a primitive-trained manipulation policy to semantically distinct tasks through explicit object-destination interfaces and closed-loop execution feedback.
| Comments: | Preprint |
| Subjects: | Robotics (cs.RO); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.04162 [cs.RO] |
| (or arXiv:2607.04162v2 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2607.04162 arXiv-issued DOI via DataCite |
Submission history
From: Ioktong Lei [view email]
[v1]
Sun, 5 Jul 2026 08:07:42 UTC (945 KB)
[v2]
Wed, 30 Sep 2026 16:37:16 UTC (631 KB)
来源：arXiv：cs.LG · arxiv.org