arXiv：cs.LG· Jiahao Yu, Song Liu, Jos\'{e} Miguel Hern\'{a}ndez-Lobato, RuiKang OuYang·· 4 小时前AI 评分34
通过 Spread Mutual Information 控制隐式生成模型中的统计依赖
Controlling Dependence in Implicit Generative Models via Spread Mutual Information
AI 导读
研究提出 Spread Mutual Information（SMI），一种对互信息在多个噪声水平上做加权积分的方法，用于控制隐式生成模型中的统计依赖。该方法通过施加扩散核（如高斯扩散）使条件与边缘密度变得平滑且严格为正，从而将基于 score 差的梯度构造扩展到原本可能奇异的分布。实验显示 SMI 在基于 MI 的方法中稳定实现有效的依赖控制，并与已有的任务专用方法保持竞争力。
正文
Abstract:Mutual information (MI) provides an objective for suppressing or encouraging statistical dependence in implicit generative models. However, direct MI evaluation is challenging in implicit models due to typically intractable densities. A remedy is estimating the generator gradient from the difference between conditional and marginal scores. This score difference can, in turn, be estimated by differentiating a log density ratio learned through classification. This construction nevertheless faces two difficulties: (i) singular distributions need not admit the required score functions, and (ii) poor overlap can hinder density-ratio estimation. We therefore introduce Spread Mutual Information (SMI), a weighted integral of MI across noise levels obtained by applying a common spreading kernel to the generated variable. Gaussian spreading yields smooth, strictly positive conditional and marginal densities, extending the gradient construction to distributions that may originally be singular. Across a variaty of experiments, SMI consistently achieves effective dependence control among MI-based methods and remains competitive with established task-specific approaches.
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.10021 [stat.ML] |
| (or arXiv:2610.10021v1 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10021 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiahao Yu [view email]
[v1]
Wed, 7 Oct 2026 13:09:22 UTC (2,733 KB)
来源：arXiv：cs.LG · arxiv.org