arXiv：cs.LG· Kasper Helverskov Petersen, Rasmus Hannibal Tirsgaard, Fran\c{c}ois R J Cornet, Mikkel Jordahn, Mikkel N. Schmidt·· 4 小时前AI 评分44
Atom-JEPA：面向 3D 原子系统的联合嵌入预测架构
Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems
AI 导读
Atom-JEPA 是一个自监督预训练框架，通过原子级和子结构级的联合嵌入预测目标，从无标注 3D 结构中学习潜在表征，并在大规模分子与晶体数据集上完成预训练。微调后，它在分子 ADMET 与量子化学性质预测任务上达到 SOTA，在晶体材料物理性质预测上也极具竞争力。代码与预训练模型 checkpoint 已公开。
正文
Abstract:Large-scale self-supervised pretraining has reshaped modern machine learning, substantially advancing the ability of language and vision models to generalize across downstream tasks. While deep learning has driven considerable progress in modeling atomistic systems in recent years, self-supervised pretraining in this domain has not yet achieved comparable downstream generalization. To address this, we introduce Atom-JEPA, a self-supervised pretraining framework that learns latent representations from unlabeled 3D structures through complementary atom-level and substructure-level objectives inspired by joint-embedding predictive architectures. We pretrain Atom-JEPA on large-scale molecular and crystalline datasets and evaluate its transfer performance by fine-tuning on a diverse set of downstream property prediction tasks. Atom-JEPA achieves state-of-the-art performance on molecular ADMET and quantum-chemical property prediction tasks, and is highly competitive in predicting the physical properties of crystalline materials. These results demonstrate the potential of latent-space predictive pretraining to support broad downstream generalization from structural data alone. Code and pretrained model checkpoints are publicly available at this https URL
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computational Physics (physics.comp-ph) |
| Cite as: | arXiv:2610.08400 [cs.LG] |
| (or arXiv:2610.08400v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.08400 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kasper Petersen [view email]
[v1]
Tue, 6 Oct 2026 14:13:51 UTC (24,425 KB)
来源：arXiv：cs.LG · arxiv.org