HuggingFace Daily Papers·· 1 天前
PDRVLA：引入物理动态表示的视觉-语言-动作模型
PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
AI 导读
PDRVLA 通过轻量级物理动态模块从视觉-语言特征中提取物理 token 来条件化动作生成，无需在线生成视频。该模块直接预测未来 RGB、深度和光流特征，并以冻结的预训练视频世界模型作辅助监督。在四个仿真基准上成功率分别为 LIBERO 99.3%、LIBERO-Plus 89.9%、RoboTwin 2.0 93.6%、SimplerEnv 81.5%。
正文 · 原文
Authors:Hanqing Wang, Zhaotian Cai, Ruichuan An, Bohan Zeng, Tao Zhang, Yilin Chen, Xu Jiang, Kaiquan Chen, Chongyang Wu, Haoran Li, Fan Yang, Mingan Lin, Wentao Zhang, Zenan Zhou, Youzheng Wu
Abstract:Vision--Language--Action (VLA) models generate robot actions from visual observations and language instructions, but their understanding of physical dynamics remains limited. Video world models can capture scene dynamics, yet generating future videos introduces high latency into the control loop. We introduce PDRVLA, whose lightweight Physical Dynamics Module extracts physical tokens from vision--language features to condition action generation without online video generation. The module learns through direct prediction of future RGB, depth, and optical-flow features, with auxiliary supervision through a frozen pretrained video world model. Training relies entirely on publicly available data, expanded through human-to-robot synthesis, subtask annotation, and embodied chain-of-thought (ECoT) annotation. Across four simulation benchmarks, PDRVLA achieves success rates of 99.3% on LIBERO, 89.9% on LIBERO-Plus, 93.6% on RoboTwin~2.0, and 81.5% on SimplerEnv. Ablations show improved future prediction with world-model supervision and evaluate how subtask conditioning, future object-pose supervision, and ECoT training affect action success.
| Subjects: | Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.37381 [cs.RO] |
| (or arXiv:2610.37381v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37381 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tao Zhang [view email]
[v1]
Tue, 27 Oct 2026 15:09:10 UTC (15,179 KB)
来源：HuggingFace Daily Papers · arxiv.org