arXiv：cs.LG· Ruslan Ageev, Huaze Tang, Runqing Miao, Jiarong Li, Wenbo Ding·· 7 小时前
DEFlow：深度平衡流最大熵强化学习
DEFlow: Deep Equilibrium Flow Maximum-Entropy Reinforcement Learning
AI 导读
DEFlow 是一种深度平衡流策略，用隐式欧拉积分替代显式欧拉采样，将全部中间动作作为平衡系统联合求解，并结合隐式微分与离散变量替换公式计算最终动作密度，从而在 soft actor-critic 中实现 off-policy 训练而无需对求解器迭代反向传播。
正文
Abstract:Expressive flow policies can represent multiple high-value actions, but sampling them with few integration steps while retaining tractable action densities remains challenging. We introduce DEFlow, a deep equilibrium flow policy that replaces explicit Euler sampling with implicit Euler integration and solves all intermediate actions jointly as an equilibrium system. We combine implicit differentiation with a discrete change-of-variables formula for the final-action density, enabling off-policy training within soft actor-critic without backpropagating through solver iterations. DEFlow achieves an average relative improvement of approximately 10% in mean successful coverage over the strongest baseline across four Multi-Goal docking variants. It also achieves the highest mean time-averaged return on four of five manipulation tasks, while remaining broadly competitive across five MuJoCo tasks. These results support implicit flow policies for tasks requiring both diversity and precision, at the cost of additional nonlinear and linear solves. Primary
| Subjects: | Machine Learning (cs.LG); Robotics (cs.RO) |
| Cite as: | arXiv:2610.16597 [cs.LG] |
| (or arXiv:2610.16597v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16597 arXiv-issued DOI via DataCite |
Submission history
From: Ruslan Ageev [view email]
[v1]
Mon, 12 Oct 2026 11:18:24 UTC (5,427 KB)
[v2]
Wed, 14 Oct 2026 06:48:06 UTC (5,427 KB)
来源：arXiv：cs.LG · arxiv.org