arXiv：cs.AI· Holger Molin, William Peng, Marco Bagatella, Randall Balestriero·· 2 天前
SpecWM：用频谱对齐的世界模型表示学习框架
World Modeling through Spectral Alignment
AI 导读
SpecWM 是一个 JEPA 框架，通过目标相似度核显式指定观测间关系，并训练编码器在嵌入几何中复现这些关系。不同核控制着表示可恢复的物理量，其中时间频谱对齐让 CEM 规划成功率在 OGBench Cube 上从 52% 提升至 68%、OGBench Scene 从 49% 提升至 77%、CALVIN 从 29% 提升至 51%。
正文
Abstract:How do we tell a world model what information to preserve? We study this question through the lens of spectral representation learning. We introduce SpecWM, a JEPA framework that explicitly specifies desired relationships between observations through a target similarity kernel and trains the encoder to reproduce those relationships in its embedding geometry. Different kernels provide different supervision, ranging from physical state information to a fully self-supervised temporal kernel requiring only the ordering of observations within trajectories. We characterize the global optima of the objective, establish conditions for linear recoverability and invariance to nuisance information, and connect temporal-kernel similarity to the behavioral successor measure under an idealized setting. Empirically, the choice of kernel strongly controls which physical quantities are recoverable from the representation. More importantly, temporal spectral alignment improves CEM planning success over LeWorldModel from 52% to 68% on OGBench Cube, 49% to 77% on OGBench Scene, and 29% to 51% on CALVIN. These results suggest that explicitly specifying representation geometry provides a simple way to steer what latent world models preserve for control.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.15429 [cs.LG] |
| (or arXiv:2610.15429v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15429 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: William Peng [view email]
[v1]
Sun, 11 Oct 2026 08:05:40 UTC (596 KB)
来源：arXiv：cs.AI · arxiv.org