arXiv：cs.LG· Arjun Subramanian·· 3 小时前AI 评分35
联合嵌入预测世界模型中的反事实动作评估、观测瓶颈与表征几何
Counterfactual Action Evaluation, Observation Bottlenecks, and Representation Geometry in Joint-Embedding Predictive World Models
AI 导读
研究提出一套评估协议，追踪同一干预在模拟器状态、栅格观测、目标嵌入与预测器输出中的传导，揭示低潜空间预测误差并不能证明世界模型能区分动作后果。在可变形物理测试台中，41.5% 的单步栅格对完全相同；579 个高可见度反事实样本的预测器响应中位数仅 0.0051 和 0.0217，方差归一化后降至 0.0027 和 0.0116。
正文
Abstract:Low latent prediction error does not establish that a world model distinguishes the consequences of its actions. We introduce an evaluation protocol that traces the same intervention through simulator state, raster observations, target embeddings, and predictor outputs. Exact simulator-state forks in a controlled deformable-physics testbed reveal distinct bottlenecks. Changed commands alter particle motion, yet 41.5% of one-step raster pairs are identical. Observation loss is not the whole explanation: among 579 high-visibility counterfactuals, median predictor-to-target response is 0.0051 and 0.0217 across two seeds, falling to 0.0027 and 0.0116 after variance normalization. An isotropic state perturbation matched to the target counterfactual embedding shift produces 190x and 53x larger predictor changes on the same visible pairs, isolating action-path under-use rather than a dead or globally shrunk predictor. MSE-only training gives 8.36x lower 10-step latent error in matched seeds, but in spectrally concentrated spaces; one VICReg target encoder is also strongly concentrated, so neither error nor rank alone certifies physical state. Finally, stiffness remains near chance even from full-resolution rasters and mechanical state while privileged material parameters decode perfectly, indicating weak identifiability under this excitation rather than encoder discard. These results motivate auditing physical effect, observation visibility, representation geometry, and action dependence separately.
| Comments: | 15 pages, 7 figures. Published in the 40th Conference on Neural Information Processing Systems (NeurIPS 2026), Workshop on Physical World AI: Geometry, Characteristics, and Multimodal Sensing. Replication Package: this https URL |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.02860 [cs.LG] |
| (or arXiv:2610.02860v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.02860 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Arjun Subramanian [view email]
[v1]
Fri, 2 Oct 2026 05:54:13 UTC (8,697 KB)
来源：arXiv：cs.LG · arxiv.org