arXiv：cs.LG· Amirparsa Bahrami, Oliver Mortensen, Mohammad Sadegh Talebi·· 4 小时前AI 评分32
递归熵风险强化学习的近最优样本复杂度：基于生成模型的分析
Near-Optimal Sample Complexity for Recursive Entropic Risk Reinforcement Learning with a Generative Model
AI 导读
研究在有限折扣 MDP 中、风险参数 β≠0 的递归熵风险偏好下，基于生成模型的价值与策略学习样本复杂度。作者对基于模型的风险敏感 Q 值迭代（MB-RS-QVI）进行精细分析，给出学习最优 Q 值函数与 ε-最优策略的 (ε,δ)-PAC 保证，其界在有效时域 1/(1-γ) 上的指数依赖优于该设定已有最优保证。
正文
Abstract:In this paper, we study the sample complexities of value and policy learning in finite discounted Markov decision processes (MDPs) under recursive entropic risk preferences with risk parameter \(\beta\neq 0\), assuming access to a generative model of the MDP. We provide a refined analysis of model-based risk-sensitive Q-value iteration (MB-RS-QVI), a plug-in model-based method introduced in prior work, and derive \((\varepsilon,\delta)\)-PAC guarantees for both learning the optimal \(Q\)-value function and an \(\varepsilon\)-optimal policy. Our bounds improve the exponential dependence on the effective horizon \(1/(1-\gamma)\) compared with the best existing guarantees for this setting. In particular, they match the existing lower bounds in their exponential dependence on \(|\beta|/(1-\gamma)\), as well as in \(S\), \(A\), \(\varepsilon\), and \(|\beta|\), up to logarithmic factors. Consequently, our analysis removes the exponential gap between the previously known upper and lower bounds, leaving only a polynomial gap in the effective horizon.
| Subjects: | Machine Learning (cs.LG); Machine Learning (stat.ML) |
| Cite as: | arXiv:2610.06931 [cs.LG] |
| (or arXiv:2610.06931v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.06931 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mohammad Sadegh Talebi [view email]
[v1]
Sat, 3 Oct 2026 01:53:13 UTC (34 KB)
来源：arXiv：cs.LG · arxiv.org