arXiv：cs.LG（机器学习，全量分类）· Morgan Byrd, Robert Wright, Sehoon Ha·· 13 小时前AI 评分33
CF-JEPA：通过可控性分解提升 JEPA 世界模型的鲁棒性
CF-JEPA: Improving Robustness of JEPA World Models via Controllability Factorization
AI 导读
CF-JEPA 是一种 JEPA 风格的世界模型，将潜在空间拆分为可控与不可控两个子空间，把干扰信息归入不可控区域，仅用控制相关的潜在信息完成任务。在 2D 和 3D 控制任务中，其常规条件下性能相当，干扰条件下表现更优，且是唯一未出现潜在坍缩的模型。该方案还在模拟机器人任务的干扰条件下得到验证。
正文
Abstract:Controlling an agent with vision requires being able to separate useful information from irrelevant background information. JEPA-style latent world models seem like a natural approach for this, as they do not perform pixel-level reconstruction; however, they are still sensitive to these distractor signals and experience latent collapse. In this work, we introduce Controllability Factorized JEPA (CF-JEPA), a JEPA-style world model which splits the latent space into controllable and uncontrollable subspaces. This factorization allows us to capture all the distractor information into the uncontrollable region, while we use the control-relevant latent information for our task. With this, we show comparable performance across 2D and 3D control tasks under nominal conditions and improved performance under distracted conditions, where CF-JEPA is the only model that does not experience latent collapse. We also validate our model under distracted conditions for a simulated robot task, highlighting the practical application of such a scheme.
| Comments: | Website: this https URL |
| Subjects: | Robotics (cs.RO); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.00727 [cs.RO] |
| (or arXiv:2610.00727v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00727 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Morgan Byrd [view email]
[v1]
Wed, 30 Sep 2026 21:18:32 UTC (404 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org