arXiv：cs.LG· Joshua Cheung, Zhang He·· 3 小时前
JEPA 世界模型研究：单步预测保真度无法可靠反映规划性能
One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models
AI 导读
一项针对端到端 JEPA 世界模型的研究发现，单步预测误差并非规划性能的可靠指标。在三个连续控制环境中，首个 epoch 将保真度提升 32%、规划仅提升 4.4 pp；此后 15 个 checkpoint 保真度再提升 15%，单步误差与规划成功率的相关性仅为 r = -0.010。
正文
Abstract:Latent world models are routinely selected, checkpointed and ablated on one-step prediction loss, on the assumption that a model which predicts the next latent better is a better model. In model-based RL this assumption is known to be unreliable; we ask whether it also fails for latent world models, where the training target is the model's own embedding and the loss is not expressed in interpretable units. We introduce a frozen-decoder protocol reporting one-step error in physical units, decomposed into the part already present in the encoder's representation and the part the predictor adds. Applying it to a recent end-to-end JEPA world model across three continuous-control environments, we move the metric by fine-tuning and measure goal-conditioned planning on the same checkpoints, with three training seeds and paired planning seeds throughout. Within one environment, a per-epoch ladder shows the relationship saturating after a single epoch: the first epoch improves fidelity by 32% and planning by 4.4 pp, but across the fifteen checkpoints that follow -- a further 15% fidelity improvement, monotone in every training seed -- the correlation between one-step error and planning success is r = -0.010. This is not a task-ceiling artifact: stratifying by a model-independent measure of training-data support, planning in the lowest-support stratum starts at 80%, leaving ample headroom, and is nonetheless flat from epoch 1 onward. Across environments the exchange rate is equally unstable: a 93% fidelity reduction on TwoRoom yields -0.7 pp of planning, while a 40% reduction on PushT yields +4.9 pp [+1.1, +8.9]. All results replicate under a decoder-free measure, latent drift. We also show that a matched continued-training control absorbs the entire fidelity gain we had attributed to an auxiliary objective.
| Comments: | 9 pages, 2 figures, 2 tables. Accepted at the NeurIPS 2026 Workshop on World Models in Physical AI (WM-PAI) |
| Subjects: | Machine Learning (cs.LG) |
| ACM classes: | I.2.6 |
| Cite as: | arXiv:2610.19375 [cs.LG] |
| (or arXiv:2610.19375v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19375 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Joshua Cheung [view email]
[v1]
Wed, 14 Oct 2026 01:19:50 UTC (109 KB)
来源：arXiv：cs.LG · arxiv.org