arXiv：cs.AI· Nikola Milosevic, Nicol\'as Hinrichs, Nico Scherf·· 18 小时前
Active Inference 可建模为凸马尔可夫决策过程
Active Inference as a Convex Markov Decision Process
AI 导读
研究将 Active Inference（AIF）的期望自由能（EFE）最小化重新表述为策略优化问题，证明在闭环控制策略下 EFE 最小化可写成凸马尔可夫决策过程（MDP）。
正文
Abstract:Active Inference (AIF) frames adaptive behavior as the minimization of expected free energy (EFE), combining epistemic and pragmatic objectives within a single variational principle. We frame AIF as policy optimization and show that, for closed-loop control policies, EFE minimization can be formulated as a convex Markov decision process (MDP). This perspective reveals that policy-dependent reward prediction errors transmit natural gradients of the expected free energy backwards in time rather than up a hierarchy. Finally, we show that coupling world-model learning with policy optimization gives active inference the structure of performative reinforcement learning. Together this places EFE minimization within modern reinforcement learning and optimization theory and opens a route toward principled algorithms for active inference.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2607.20152 [cs.LG] |
| (or arXiv:2607.20152v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2607.20152 arXiv-issued DOI via DataCite |
Submission history
From: Nikola Milosevic [view email]
[v1]
Wed, 22 Jul 2026 13:52:40 UTC (308 KB)
[v2]
Mon, 21 Sep 2026 10:39:39 UTC (314 KB)
[v3]
Tue, 13 Oct 2026 13:55:26 UTC (309 KB)
来源：arXiv：cs.AI · arxiv.org