arXiv：cs.LG· Yuxiang Fu, Qi Yan, Zike Wu, Yongxing Zhang, Purang Abolmaesumi, Lele Wang, Renjie Liao·· 4 小时前AI 评分47
无需数据的扩散模型蒸馏：一致分布匹配方法
Consistent Distribution Matching for Data-Free Diffusion Distillation
AI 导读
研究者提出 Consistent Distribution Matching，一种无需数据、无需模拟的扩散与流模型蒸馏方法，仅用冻结教师模型和可训练学生模型两个模型、优化单一目标，即可实现一步或少步生成。
正文
Abstract:Flow and diffusion models suffer from slow inference due to computationally expensive numerical integration. Distillation provides a promising way for a student model to learn from a teacher's dynamics, enabling one-step or few-step generation. However, existing methods often depend on curated distillation datasets, costly teacher rollouts, or auxiliary proxy networks, which complicate model training and scaling. In this work, we propose Consistent Distribution Matching, a simulation-free and data-free distillation method for accelerating diffusion and flow models while preserving strong generative capacity. Our key insight is to unify sample generation and score estimation with one student network. Thus, our framework uses only two models, a frozen teacher and a trainable student, and optimizes one objective. We prove that minimizing our objective indicates Wasserstein convergence of the student flow-map pushforwards to the teacher marginals. On ImageNet 256$\times$256, our method attains an FID of 2.04 with a single function evaluation (1-NFE) and a 4-NFE FID of 1.37 within 40 epochs of training, surpassing the state-of-the-art distillation baselines without data. Our code code and model are available at this https URL.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.09221 [cs.LG] |
| (or arXiv:2610.09221v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09221 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuxiang Fu [view email]
[v1]
Tue, 6 Oct 2026 23:34:13 UTC (44,948 KB)
来源：arXiv：cs.LG · arxiv.org