HuggingFace Daily Papers·· 2 天前
State World Models：分离规则执行与视频渲染，实现有状态的世界交互
State World Models: Execute the Rules, Generate the Pixels
AI 导读
研究提出 Stateful Harness 框架，将世界状态演化与视频渲染分离，用冻结的视频模型渲染事件并根据生成片段更新状态以支持规则驱动的交互。团队还构建 WorldState-Bench，含 50 对仅隐藏状态不同的反事实样本对。
正文 · 原文
Abstract:Existing world models can generate action-conditioned video from a single image. Many focus on camera movement, allowing users to explore the generated scene. However, navigation alone does not ensure that an action follows persistent rules when its outcome depends on state that is invisible in the current frame. We present Stateful Harness, a framework that separates world evolution from video rendering to support stateful and rule-governed interaction. It maintains an evolving world state under specified rules, renders each resulting event with a frozen video model, and uses the generated clip to update the world state for the next turn. At each turn, the harness resolves the action selected by the user and compiles the resulting event into an instruction, while a preview clip from an editable 3D scene provides spatial and camera guidance. To evaluate rule compliance, we build WorldState-Bench, a benchmark of 50 counterfactual twin pairs that share an initial frame and action but differ in hidden state and required outcome. Under rule-only prompting, the best of the 11 evaluated video backends produces the required endings for both cases in 32% of pairs. For MiniMax-H3, specifying the required endings raises this rate from 32% to 64%, while the full harness reaches 92% with the same frozen model. These results support separating rule execution from video rendering for world interactions with hidden state.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.26811 [cs.CV] |
| (or arXiv:2610.26811v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.26811 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yanran Zhang [view email]
[v1]
Mon, 19 Oct 2026 17:50:32 UTC (38,821 KB)
来源：HuggingFace Daily Papers · arxiv.org