arXiv：cs.LG（机器学习，全量分类）· Hanna Malet, Gabriel Turinici·· 15 小时前AI 评分38
扩散模型中的特征选择性模型崩溃：完全替换与固定预算训练对比
Feature Selective Model Collapse in Diffusion Models: Total Replacement versus Fixed-Budget Training
AI 导读
研究对比了扩散模型在合成数据训练下的两种协议：完全替换真实数据会导致数据集快速退化，而固定预算训练（保留全部历史数据但每代仅采样固定规模）仅造成部分退化，能保留部分特征。
正文
Abstract:Model collapse arises when generative models are trained on synthetic data produced by earlier models. The phenomenon has attracted considerable attention because of its societal and technical implications. However, previous studies have reached seemingly contradictory conclusions: replacing real data with synthetic data causes collapse (Shumailov et al.), yet accumulating real data alongside synthetic data can prevent it. For diffusion models, we study an intermediate regime typical of finite-budget pipelines: all past datasets and the real data are kept, but each new model is trained on a fixed-size sample from this growing pool, so the real fraction vanishes without any data being removed. Experiments on a 2D spiral dataset as well as the image benchmarks (MNIST, Fashion-MNIST, and CIFAR-10) show that replacement protocol degrades dataset rapidly as in the literature, whereas the fixed budget degrades only partially, sparing some features. A linear-response model of the multi-generational parameter dynamics, analyzed by stochastic recursion, confirms that the two protocols differ: some features will be fragile and lost within a few generations for both protocols, while some will be robust and preserved over practically unbounded horizons under the fixed budget protocol.
| Comments: | Neurips 2026 PriGM workshop paper |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Statistics Theory (math.ST) |
| ACM classes: | I.2.6; G.3; F.2.2 |
| Cite as: | arXiv:2610.01318 [cs.LG] |
| (or arXiv:2610.01318v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.01318 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Gabriel Turinici [view email]
[v1]
Thu, 1 Oct 2026 08:47:48 UTC (526 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org