arXiv：cs.LG· Hong Huang, Chenhongyi Yang, Junzhe Sun, Animesh Sinha, Wuyang Chen, Yifan Jiang·· 4 小时前
Omni-Diffusion-Distill：统一多模态扩散大语言模型的少步蒸馏
Omni-Diffusion-Distill: Few-Step Distillation of Unified Multimodal Diffusion Large Language Models
AI 导读
研究者提出 Omni-Diffusion-Distill 两阶段蒸馏框架，将统一多模态扩散大语言模型的图像生成从 128 步降至 8 步、多模态理解从 512 步降至 64 步，分别实现 18.2x 和 21.2x 的墙钟加速。
正文
Abstract:Unified multimodal diffusion large language models (dLLMs) offer a single architecture for both image generation and multimodal understanding, but their iterative decoding requires tens to hundreds of forward passes. Existing few-step distillation methods largely focus on either image generation or text generation, making it unclear how to compress a fully discrete multimodal dLLM into a single efficient student while preserving both generation and understanding. We introduce Omni-Diffusion-Distill, a unified two-stage distillation framework that retains strong generation and understanding capabilities while substantially reducing the inference cost of a unified multimodal dLLM. Omni-Diffusion-Distill aligns the distillation of both generation and understanding, for both images and text, in the discrete token space. In the first stage, the student is trained to skip decoding steps by replaying cached teacher trajectories, and in the second stage the student is refined on intermediate states along its own rollouts. We further remedy two sources of degradation in unified distillation with a pairwise collision penalty that reduces repetition under parallel text decoding, and entropy-matched guidance that prevents entropy collapse caused by fitting the sharpened teacher distribution in image generation. Omni-Diffusion-Distill achieves state-of-the-art trade-offs between decoding efficiency and generation and understanding performance for multimodal dLLMs, reducing image generation from 128 to 8 decoding steps and multimodal understanding from 512 to 64, giving 18.2x and 21.2x wall-clock speedups. Under these budgets, it scores 0.828 on GenEval and 83.0 on DPG-Bench for text-to-image generation, while reaching GPT judge scores of 20.0 on MM-Vet and 57.2 on COCO captioning (twice the teacher's 28.4 at the same steps) for multimodal understanding.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.10990 [cs.CV] |
| (or arXiv:2610.10990v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10990 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hong Huang [view email]
[v1]
Wed, 7 Oct 2026 23:24:16 UTC (7,611 KB)
来源：arXiv：cs.LG · arxiv.org