arXiv：cs.LG· Wenbin Zhou, Michael Lingzhi Li, Shixiang Zhu·· 4 小时前AI 评分29
Safe Meta-Policy Design with Risk Control：带风险控制的策略更新调度方法
Safe Meta-Policy Design with Risk Control
AI 导读
研究提出一种离线元策略方法，在训练未来候选模型之前规划策略更新时机，在提升收益与控制性能退化风险之间取得平衡。方法将更新计划建模为有向无环图中的路径，并用动态规划在"更新次数劣于被替换策略"的期望预算约束下最大化期望累积价值，从历史学习轨迹估计切换的价值与风险。一阶分析表明策略改进的信噪比是决定更新频率、等待时间与风险分配的关键因素，合成数据与临床试验数据实验展示了性能—风险权衡并对比了基线方法。
正文
Abstract:Models can be retrained as new data arrive, but deploying every new version risks replacing a good policy with a worse one. We study how to plan policy updates (i.e., meta-policy) before future candidates are trained, balancing the benefits of improvement against the risk of performance regression. Our offline meta-policy maximizes expected cumulative value subject to a budget on the expected number of updates that perform worse than the policies they replace. We estimate the value and risk of possible switches from historical learning trajectories, represent an update schedule as a path in a directed acyclic graph, and select a schedule using dynamic programming. A leading-order analysis identifies the signal-to-noise ratio of policy improvement as a key driver of update frequency, waiting times, and risk allocation: clearer improvements support earlier, more frequent updates, while noisier improvements call for longer waits or greater risk expenditure. Their asymptotic rates also reveal a diminishing marginal cost of achieving greater safety over time. Experiments on synthetic and clinical trial data illustrate the performance--risk tradeoff and compare our method with alternative baselines.
| Subjects: | Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC) |
| Cite as: | arXiv:2610.10393 [stat.ML] |
| (or arXiv:2610.10393v1 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10393 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wenbin Zhou [view email]
[v1]
Wed, 7 Oct 2026 16:48:38 UTC (966 KB)
来源：arXiv：cs.LG · arxiv.org