arXiv：cs.LG· Wen Ge, Lei Jiang, Lingyi Yang, Leonardo Roveri, Mingxuan Yi, Po-Yu Chen, Niels Cariou-Kotlarek, Francois Buet-Golfouse, Gaurav Mittal, Hao Ni·· 10 小时前
Sig-DEG：让扩散模型蒸馏更快更轻
Sig-DEG for Distillation: Making Diffusion Models Faster and Lighter
AI 导读
Sig-DEG（Signature-based Differential Equation Generator）是一种用于蒸馏预训练扩散模型的生成器，结合部分签名与循环结构近似随机微分方程解。
正文
Abstract:Diffusion models achieve state-of-the-art generative performance but remain computationally expensive at inference, often requiring many discretization steps. We propose Sig-DEG (Signature-based Differential Equation Generator), a generator for distilling pre-trained diffusion models. Inspired by high-order stochastic Taylor expansions, Sig-DEG combines partial signatures that summarize Brownian motion over subintervals with a recurrent structure to approximate stochastic differential equation solutions to achieve universality. We formulate distillation as supervised learning, matching fine-resolution teacher trajectories on a coarse time grid. At inference, Sig-DEG enables fast generation, as the partial signatures can be simulated exactly from their known distributions without requiring fine-grained Brownian paths. Experiments on image and time-series generation demonstrate competitive sample quality with inference speedups of up to several hundred times over the teacher. A Schrödinger bridge example further demonstrates Sig-DEG's applicability beyond diffusion distillation.
| Comments: | 44 pages. Revised version with expanded theoretical analysis, updated experiments, and revised presentation |
| Subjects: | Machine Learning (cs.LG); Probability (math.PR); Machine Learning (stat.ML) |
| Cite as: | arXiv:2508.16939 [cs.LG] |
| (or arXiv:2508.16939v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2508.16939 arXiv-issued DOI via DataCite |
Submission history
From: Wen Ge Miss [view email]
[v1]
Sat, 23 Aug 2025 08:16:14 UTC (16,978 KB)
[v2]
Tue, 13 Oct 2026 19:33:10 UTC (2,595 KB)
来源：arXiv：cs.LG · arxiv.org