视频扩散模型内部编码物理合理性，准确率约 81.27% 超越 V-JEPA 与 VideoMAE

View PDF HTML (experimental)

Abstract:Modern video diffusion models generate increasingly realistic and temporally coherent videos, motivating their use as candidate world simulators. Yet it remains unclear whether these models internally encode physical structure, or merely reproduce motion patterns seen during training. We study this question by probing video diffusion models along latent trajectories corresponding to real videos with known physical plausibility. To obtain such trajectories, we approximately invert the deterministic sampling process by integrating the learned velocity field backward from a clean video latent to noise, giving access to the model's intermediate states and attention maps. Using these recovered trajectories, we show that physical plausibility is linearly decodable from diffusion transformer states across IntPhys and InfLevel, reaching around 81.27% average accuracy and outperforming dedicated representation-learning baselines such as V-JEPA and VideoMAE. Surprisingly, this signal is absent from the VAE latent input and emerges inside the denoising transformer itself, despite the model not being trained with a self-supervised predictive objective. These findings suggest that physically meaningful representations can arise as a byproduct of generative denoising.

Subjects:	Graphics (cs.GR); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
Cite as:	arXiv:2606.05328 [cs.GR]
	(or arXiv:2606.05328v2 [cs.GR] for this version)
	https://doi.org/10.48550/arXiv.2606.05328 arXiv-issued DOI via DataCite

Submission history

From: Parsa Esmati [view email]
[v1] Wed, 3 Jun 2026 18:11:51 UTC (16,702 KB)
[v2] Fri, 9 Oct 2026 09:38:45 UTC (16,720 KB)