arXiv：cs.LG· Jiechen Jackie Zhang, O. Deniz Akyildiz·· 13 小时前
基于粒子的广义随机优化：面向不可解梯度损失函数的扩散式方法
Particle-based Generalised Stochastic Optimisation
AI 导读
研究者提出一类基于扩散的随机粒子优化方法，面向损失梯度为参数依赖分布积分、梯度不可解的问题，涵盖生成模型训练、微调和隐变量模型学习。该方法引入平均场动力学及其交互粒子近似，可将多种现有算法纳入统一框架。在适定性与联合收缩性假设下，作者证明了指数收敛，并给出连续时间粒子系统的非渐近误差界，还构造了高阶 Langevin 与动量优化器，在最大边缘似然估计和基于能量的模型训练上进行了评估。
正文
Abstract:We develop a class of general diffusion-based stochastic particle optimisation methods for loss functions with intractable gradients. Specifically, we consider problems in which the loss gradient is an integral with respect to a parameter-dependent distribution, a structure that includes training generative models, fine-tuning, and learning latent-variable models. We introduce mean-field dynamics and its interacting-particle approximations, which contain several existing algorithms as special cases and provide a route to constructing new methods. Under well-posedness and joint contractivity assumptions, we prove exponential convergence and show that the continuous-time particle system admits a non-asymptotic error bound. We illustrate our framework by developing higher-order Langevin and momentum optimisers and evaluating them on maximum marginal-likelihood estimation and energy-based-model training.
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG); Computation (stat.CO) |
| Cite as: | arXiv:2608.02844 [stat.ML] |
| (or arXiv:2608.02844v2 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2608.02844 arXiv-issued DOI via DataCite |
Submission history
From: Jiechen Jackie Zhang [view email]
[v1]
Mon, 3 Aug 2026 19:59:47 UTC (6,185 KB)
[v2]
Fri, 9 Oct 2026 17:01:27 UTC (4,997 KB)
来源：arXiv：cs.LG · arxiv.org