arXiv：cs.AI· Zeyuan Wang, Da Li, Yulin Chen, Yuehu Gong, Yanming Guo, Liang Bai, Tianyuan Yu, Yanwei Fu·· 13 小时前
Stochastic MeanFlow Policies（SMFP）：一步生成式控制策略实现熵正则镜像下降
Stochastic MeanFlow Policies: One-Step Generative Control with Entropic Mirror Descent
AI 导读
研究者提出 Stochastic MeanFlow Policies（SMFP），一种将 MeanFlow 噪声到动作映射与高斯重参数化结合的一步生成式策略类，可通过可学习的条件高斯尺度提供解析熵下界，从而在一步策略上实现熵正则化镜像下降学习。在七个 MuJoCo 基准上，SMFP 以单步推理取得优于高斯及生成式基线的表现。额外实验验证了其多模态动作建模能力与稀疏奖励导航中的有效探索。
正文
Abstract:Combining entropy regularisation with mirror descent in online off-policy reinforcement learning requires expressive policies with efficient sampling and explicit entropy control. Gaussian policies struggle to represent multimodal targets, whereas generative policies often require iterative sampling or lack tractable entropy estimates. We introduce Stochastic MeanFlow Policies (SMFP), a one-step generative policy class that combines MeanFlow-based noise-to-action mappings with Gaussian reparameterisation. The latent mapping captures diverse action modes, while a learnable conditional Gaussian scale provides an analytic entropy lower bound. We use this bound to regulate stochasticity alongside direct value guidance and advantage-weighted MeanFlow regression, enabling entropy-regularised mirror-descent learning with one-step policies. Across seven MuJoCo benchmarks, SMFP achieves strong performance against Gaussian and generative baselines with single-step inference. Additional experiments demonstrate multimodal action modelling and effective exploration in sparse-reward navigation.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.21282 [cs.LG] |
| (or arXiv:2605.21282v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.21282 arXiv-issued DOI via DataCite |
Submission history
From: Zeyuan Wang [view email]
[v1]
Wed, 20 May 2026 15:14:14 UTC (638 KB)
[v2]
Thu, 21 May 2026 06:09:02 UTC (638 KB)
[v3]
Fri, 9 Oct 2026 02:37:42 UTC (716 KB)
来源：arXiv：cs.AI · arxiv.org