arXiv：cs.LG（机器学习，全量分类）· Maksim Bobrin, Maksim Zhdanov, Dmitry Dylov·· 1 天前AI 评分39
Fenchel Tilting：生成模型高效微调的加权校正方法
Fenchel Tilting: Weighted Correction for Efficient Finetuning of Generative Models
AI 导读
研究者提出 Fenchel Tilt Flow Control（FTFC），将效用优化与生成模型拟合解耦，先在预训练样本上联合拟合有效奖励与密度比权重，再冻结权重以单阶段重要性加权去噪或流匹配修改扩散/流模型，无需对采样轨迹求导。在图像与分子生成基准上，FTFC 在多种偏好函数下优于基线，效率最高提升 20 倍。
正文
Abstract:Adapting a pretrained generative model to an arbitrary preference expressed as a utility function underlies reward alignment, guided design, and constraint satisfaction, enabling diverse applications. Existing fine-tuning methods trade off generality against computational cost: they either restrict the family class of supported preferences to keep optimization simple or preserve generality at the expense of efficiency. We introduce Fenchel Tilt Flow Control (FTFC), which decouples utility optimization from generative-model fitting. FTFC first optimizes for a target distribution by jointly fitting an effective reward and density-ratio weights on pretrained samples. Method combines the utility's variational structure with Fenchel duality, supporting general $f$-divergence penalties that determine how rewards are transformed into an distribution-correction weights. These weights are then frozen and used to modify a diffusion or flow model in a single stage of importance-weighted denoising or flow matching, without differentiating through sampling trajectories. We establish exact duality for concave utilities under suitable conditions and show that weighted fitting reproduces the optimal target distribution for a given utility. Across image and molecule generation benchmarks, FTFC improves over baselines on diverse preference functions, while also being up to $20\times$ more efficient. roposed method enables adaptation beyond expected-reward maximization without complex optimization, while preserving robustness for more general class of the utility functions compared to baselines.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.40030 [cs.LG] |
| (or arXiv:2609.40030v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.40030 arXiv-issued DOI via DataCite |
Submission history
From: Maksim Bobrin [view email]
[v1]
Wed, 30 Sep 2026 16:02:58 UTC (13,919 KB)
[v2]
Thu, 1 Oct 2026 07:56:18 UTC (13,919 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org