HuggingFace Daily Papers·· 2 天前
RoboValue：面向机器人价值模型统一评估的细粒度仿真与真实基准
RoboValue: A Fine-Grained Sim-and-Real Benchmark for Unified Evaluation of Robotic Value Models
AI 导读
研究团队提出 RoboValue，一个通过共享接口从任务状态理解、时序进度监控、失败与恢复推理、价值一致性四个维度统一评估机器人价值模型的细粒度仿真与真实基准。其数据集覆盖 15 个仿真和 20 个真实世界操作任务，提供专家演示与标注轨迹。零样本和单样本评测显示，即使结果与前进进度得分较强的模型，也在细粒度指令接地、依赖历史的进度跟踪和恢复评估上出错。
正文 · 原文
Authors:Shengbang Liu, Zhengye Du, Zhilong Wan, Honghao Su, Chenxiang Xia, Chang Ge, Jinyang Xiao, Nan Wang, Zhipeng Hu, Huining Yuan, Shu'ang Yu, Yue Chen, Tianxing Chen, Chuankang Li, Maoqing Yao, Wenbo Ding, Yu Wang, Guanghui Ren, Chao Yu
Abstract:Robotic value models assess task execution from visual observations and instructions, providing feedback for policy learning and execution monitoring. However, existing evaluations cover only selected aspects of execution understanding, leaving the reliability of value feedback insufficiently tested. We introduce RoboValue, a fine-grained sim-and-real benchmark for unified evaluation of robotic value models. Its framework evaluates four complementary capability dimensions through shared interfaces: task-state understanding, temporal progress monitoring, failure and recovery reasoning, and value consistency. Diagnostic trajectories and instruction variations test whether value judgments accurately reflect task requirements and changes throughout execution. RoboValue-Dataset covers 15 simulation and 20 real-world manipulation tasks, providing expert demonstrations for training and annotated trajectories for fine-grained evaluation. We evaluate representative robotic value models in zero-shot and one-shot settings under standard conditions and separate shifts in robot embodiment and environment. Our results reveal errors in fine-grained instruction grounding, history-dependent progress tracking, and recovery assessment, even in models with strong outcome and forward-progress scores. The same semantic subtask can also receive substantially different value gains across valid execution orders. These findings identify concrete targets for value supervision, while the dataset and public leaderboard support continued assessment of feedback reliability. The website is available at this https URL.
| Subjects: | Robotics (cs.RO) |
| Cite as: | arXiv:2610.26663 [cs.RO] |
| (or arXiv:2610.26663v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.26663 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shengbang Liu [view email]
[v1]
Mon, 19 Oct 2026 16:19:08 UTC (27,094 KB)
来源：HuggingFace Daily Papers · arxiv.org