arXiv：cs.LG· Jonas Kneifl, Jakub Skalski, Bart{\l}omiej Twardowski, Kamil Deja·· 3 小时前AI 评分44
物理量是否存在于激活中？在视频扩散模型中定位物理量
Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Models
AI 导读
研究者通过探测视频 Diffusion Transformer（DiT）的内部表示，发现模拟器生成的运动学与刚体动力学物理量在去噪过程早期即可被线性高精度解码，且显著优于直接从模型自身加噪 latent 解码的基线。
正文
Abstract:Video generation models produce strikingly realistic sequences and are increasingly proposed as world models, yet recent benchmarks reveal pronounced deficits in their physical reasoning. This raises the question of whether these models internalize physical principles or merely reproduce familiar motion patterns. We address this by probing internal representations of video Diffusion Transformers (DiTs) for simulator-derived ground-truth physical quantities spanning kinematic motion and rigid-body dynamics under gravity and contact. We find that these quantities are linearly decodable with high accuracy early in the denoising process, substantially outperforming a baseline decoded directly from the model's own noised latents, indicating that the relevant physical information is actively constructed during denoising rather than already present in the input. Additionally, we show that activations at on-object tokens carry the relevant physical information and that quantities defined over multiple frames are readable from single latent frames. Hence, information is sharply localized within the token sequence and is computed globally but stored locally. The probes further show partial extrapolation, transferring to scene variations and object configurations outside their training regime, so what they read is not simply a correlate of the scenes they were fit on. When fitted directly in the full-resolution activation space, the probing directions can serve as steering vectors to change the model's output.
| Comments: | 22 pages, 8 figures, 5 tables |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.03154 [cs.CV] |
| (or arXiv:2610.03154v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03154 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jonas Kneifl [view email]
[v1]
Fri, 2 Oct 2026 11:26:09 UTC (3,844 KB)
来源：arXiv：cs.LG · arxiv.org