SD-JEPA：将潜在世界模型分解为进展与内容子空间

View PDF HTML (experimental)

Abstract:World models used for planning learn to predict how a scene evolves, yet they rarely hold a global view of how far a task has progressed. We argue that such a view should be part of the representation, and that it can be compact: describing a scene takes many dimensions, but where the scene stands on its way to the goal can be summarised by only a few. We propose SD-JEPA, a joint-embedding predictive world model whose latent is split into a small progression subspace and a large content subspace. A temporal triplet loss keeps consecutive frames of a trajectory close in the progression coordinates and separates different trajectories, while a distributional regulariser keeps the content coordinates from collapsing. On four control benchmarks, the model plans as well as the same architecture without the split. In return, its latent becomes easier to read. A few percent of its dimensions are enough to predict progress within an episode, and the angle they form follows the task, moves backward when the agent backtracks and changes fastest around events such as grasps and releases. Surprise also becomes anchored in the trajectory: when the scene changes mid-episode, the angle spikes and then relocalizes on a new part of the progression, anchoring surprise in the agent's own map of the task.

Comments:	Rewritten presentation, corrected method description, new figures, updated related work
Subjects:	Machine Learning (cs.LG)
Cite as:	arXiv:2605.31111 [cs.LG]
	(or arXiv:2605.31111v3 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2605.31111 arXiv-issued DOI via DataCite

Submission history

From: Lucas Thil [view email]
[v1] Fri, 29 May 2026 10:23:48 UTC (2,536 KB)
[v2] Tue, 15 Sep 2026 19:52:43 UTC (2,536 KB)
[v3] Thu, 8 Oct 2026 19:10:10 UTC (2,685 KB)