arXiv：cs.LG（机器学习，全量分类）· Shixuan Liu, Joan Serr\`a, Kin Wai Cheuk, Jinju Kim, Woosung Choi, Yukara Ikemiya, Wei-Hsiang Liao, Jiaqi W. Ma, Yuki Mitsufuji·· 5 小时前AI 评分34
扩散模型训练数据归因新方法 TID 与 TIDE：蒸馏分数差异实现毫秒级归因
Distilling Diffusion Score Discrepancy for Efficient Training Data Attribution
AI 导读
研究者提出 TID，用局部分数差异直接衡量扩散模型训练样本对生成结果的影响，适用于 DDPM、EDM 和 flow matching，无需重训练即可估计。将 TID 蒸馏为前向学生模型 TIDE 后，在 CIFAR-10、ArtBench-10 和 MS-COCO 的反事实评测中保持大部分精度，单次查询成本降低四到五个数量级，毫秒级完成归因，快于生成本身。
正文
Abstract:Training data attribution for diffusion models aims to identify the training samples that influence a generated instance, but existing methods either require costly per-sample gradient computation or query-specific model optimization. Moreover, most methods attribute changes in a proxy loss rather than changes in the actual model's generative behavior. We address these limitations by formulating attribution directly with a local score discrepancy measure, which applies to any diffusion variant (including DDPM, EDM, and flow matching), and by showing that such measure can be estimated without retraining, as a preconditioned gradient similarity. We instantiate this estimator as Training-data Influence via score Discrepancy (TID), which uses Kronecker-factored curvature to avoid random projections and per-sample gradient storage. We then distill TID into TIDE, a forward-only student trained online to reproduce the teacher's rankings from the diffusion model's internal activations. Under counterfactual evaluation on CIFAR-10, ArtBench-10, and MS-COCO, TID matches or outperforms state-of-the-art approaches, while TIDE retains most of TID's accuracy at four to five orders of magnitude lower per-query cost, attributing generated samples in milliseconds and faster than the generation itself.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.38776 [cs.LG] |
| (or arXiv:2609.38776v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.38776 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shixuan Liu [view email]
[v1]
Wed, 30 Sep 2026 02:04:00 UTC (2,095 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org