arXiv：cs.LG（机器学习，全量分类）· Zheyuan Zhang, Suyu Ye, Nakul Agarwal, Hossein Nourkhiz Mahjoub, Ehsan Moradi Pari, Daniel Khashabi, Tianmin Shu, Vaishnav Tadiparthi·· 15 小时前AI 评分38
JEPA-TTT：面向动态变化下规划的潜在世界模型持久测试时训练
JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts
AI 导读
JEPA-TTT 在测试阶段持续自适应预训练动作条件 JEPA 世界模型的潜在动态预测器，视觉编码器与奖励头保持冻结，自监督更新跨回合累积。方法采用密集回放，在每个时间偏移构建预测窗口并存入增长缓冲区采样更新。在 4 个连续控制环境的 8 种动态变化下，500 个测试回合后自回归潜在预测误差平均降低 83%，规划性能较冻结 JEPA 世界模型提升 153%。
正文
Abstract:World models enable agents to plan by predicting future states of the environment, but their predictions can become unreliable when test-time dynamics differ from those seen during training. We present JEPA-TTT, which adapts the latent dynamics predictor of a pretrained action-conditioned Joint-Embedding Predictive Architecture world model throughout test time. Self-supervised updates accumulate across episodes, while the visual encoder and reward head remain fixed, preserving the pretrained representation and task objective. Planning requires neither a goal image nor online environment reward. JEPA-TTT uses dense replay, which forms prediction windows at every temporal offset, retains them in a growing buffer, and samples minibatches from that buffer for predictor updates. Across eight dynamics shifts in four continuous-control environments, JEPA-TTT improves planning on every shift. After 500 test-time episodes, it reduces autoregressive latent prediction error by 83% on average and improves planning performance by 153% over the frozen JEPA world model. These results show that persistent self-supervised test-time training can adapt a pretrained latent world model under changed dynamics.
| Comments: | Accepted to World Models in Physical AI Workshop @ NeurIPS 2026 | Project page: this https URL |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.00722 [cs.LG] |
| (or arXiv:2610.00722v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00722 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zheyuan Zhang [view email]
[v1]
Wed, 30 Sep 2026 21:13:59 UTC (3,174 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org