ALeWM：面向世界模型的自适应潜空间容量方法

View PDF HTML (experimental)

Abstract:We introduce Adaptive LeWorldModel (ALeWM), a world model based on a joint-embedding predictive architecture (JEPA) that learns to concentrate predictive information in compact prefixes of a wide latent representation. To encourage this ordering, ALeWM learns a sequence-conditioned distribution over prefix lengths and trains the predictor to estimate the full next embedding from a sampled input prefix. As standard anti-collapse objectives encourage variation across latent coordinates and do not organize them by predictive importance, we also introduce MixSIGReg. MixSIGReg regularizes the masked embeddings against a prior-weighted mixture with Gaussian active prefixes and zeros in the remaining coordinates. As a result, the ALeWM objective encourages early coordinates to retain information useful for prediction and recursive planning. Our analysis shows that the mixture distribution used by MixSIGReg assigns higher variance to earlier coordinate blocks and lower variance to later ones. In addition, we show that, under specified assumptions, prediction error is minimized by placing the information most useful for prediction in earlier blocks. Empirically, we study the behavior of ALeWM in a controlled dynamical system with known state variables and in goal-conditioned visual control. We show that ALeWM consistently achieves higher mean success rates than tuned fixed-width LeWM, with lower planning capacity on average.

Subjects:	Machine Learning (cs.LG)
Cite as:	arXiv:2609.32921 [cs.LG]
	(or arXiv:2609.32921v2 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2609.32921 arXiv-issued DOI via DataCite

Submission history

From: Idan Achituve [view email]
[v1] Sat, 26 Sep 2026 20:16:33 UTC (3,141 KB)
[v2] Tue, 6 Oct 2026 06:24:43 UTC (3,143 KB)