arXiv：cs.AI· Yuanpeng He, Ceyao Zhang, Fangjing Li, Kefei Zhu, Lijian Li, Yueheng Li, Tianjia He, Wenpin Jiao, Zhi Jin, Yaodong Yang, Yuanpei Chen·· 6 小时前
RAP-WAM：通过预测排练与动作扰动学习世界-动作模型
RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations
AI 导读
RAP-WAM 将预测排练（Predictive Rehearsal）与动作扰动（Action Perturbations）结合未来监督查询和耦合动作投影，从视频预训练骨干学习世界-动作模型，无需单独评分网络或部署时探测。
正文
Authors:Yuanpeng He, Ceyao Zhang, Fangjing Li, Kefei Zhu, Lijian Li, Yueheng Li, Tianjia He, Wenpin Jiao, Zhi Jin, Yaodong Yang, Yuanpei Chen
Abstract:Recent world action models draw on video generation priors to jointly predict future observations and robot actions. Adapting these priors to control requires determining how future predictions should guide policy learning. RAP-WAM combines predictive Rehearsal and Action Perturbations (RAP) with future-supervised queries and Coupled Action Projection for policy learning. Predictive rehearsal trains on model-generated inputs while retaining recorded targets, and accumulates recurring errors into a spatial prior. We perturb actions and measure changes in video predictions. Action-response-based spatiotemporal credit assignment combines these responses, prediction error, and the rehearsal prior to allocate visual supervision; spatially aggregating the responses determines supervision intensity across future action steps. No separate scoring network or deployment-time probe is required. The base learns from a video-pretrained backbone without intermediate pretraining on external embodied data. We evaluate the fixed base and its agent-refined, task-conditioned system. On 42 RoboDojo tasks, their average success rates are 12.85% and 15.62%, with Scores of 17.86 and 20.59. Across 15 real-robot tasks, the complete system achieves 89.00% task-macro-average success, exceeding the strongest of five baselines by 37.67 percentage points. Across 315 fixed-output simulation windows, 45.21% of response-top20 positions fall outside error-top20, revealing information beyond prediction error.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16443 [cs.AI] |
| (or arXiv:2610.16443v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16443 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuanpeng He [view email]
[v1]
Mon, 12 Oct 2026 09:09:21 UTC (11,608 KB)
来源：arXiv：cs.AI · arxiv.org