HuggingFace Daily Papers·· 1 天前
Lab2Sim：把真实实验视频转成可交互虚拟实验的基准与 EduVideo2Code 方法
Lab2Sim: Turning Real-World Experiment Videos into Interactive Virtual Experiments
AI 导读
Lab2Sim 是一个包含 218 个科学实验视频、带证据支撑规格与 11,881 条评分细则的基准，用于把实验视频转成可交互虚拟实验。配套方法 EduVideo2Code 分层提取实验流程、编码为显式状态机指导代码生成，并通过执行验证迭代页面。
正文 · 原文
Abstract:Science experiment videos demonstrate how experiments are performed, but offer no direct interaction for learners. Turning them into virtual experiments requires not only reproducing the observed apparatus and state changes, but also enforcing the preconditions governing each operation. We introduce Lab2Sim, a benchmark of 218 science experiment videos with evidence-grounded specifications and 11,881 rubric items. To address this task, we propose EduVideo2Code, which extracts experimental procedures hierarchically, encodes them as explicit state machines to guide code generation, and refines the resulting pages through execution-based verification. Experiments show that existing multimodal models readily generate runnable pages but struggle to faithfully reconstruct experimental apparatus. EduVideo2Code consistently improves all four evaluation dimensions across two backbone models, with apparatus fidelity gains of 24.15 points.
| Comments: | 21 pages, 9 figures |
| Subjects: | Multimedia (cs.MM) |
| Cite as: | arXiv:2611.02928 [cs.MM] |
| (or arXiv:2611.02928v1 [cs.MM] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02928 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xusen Hei [view email]
[v1]
Mon, 2 Nov 2026 10:56:06 UTC (4,872 KB)
来源：HuggingFace Daily Papers · arxiv.org