arXiv：cs.AI· Itai Zilberstein, Ioannis Anagnostides, Zachary W Sollie, Arman Kilic, Tuomas Sandholm·· 13 小时前
面向策略优化的偏好引出方法及其在心脏移植与人类价值对齐中的应用
Preference Elicitation for Policy Optimization and Application to Aligning Heart Transplantation with Human Values
AI 导读
研究者提出一种面向线性效用的偏好引出算法，直接针对分配结果而非算法决策收集偏好，分两阶段：先用成对比较学习割平面快速收缩属性权重空间，再收敛到用户效用函数。该方法用于心脏移植分配，通过学习并聚合与人类价值对齐的效用函数优化策略，相比现状策略 0.66 的竞争比，其方法达到近最优的 0.95；在用户研究中，基于自然语言解释的 LLM 方法准确率均未超过学到的线性效用。
正文
Abstract:Preference elicitation is essential for aligning AI systems with human values. Prior approaches (e.g., for organ allocation) often ask stakeholders to compare the decisions of an algorithm (e.g., patient A vs. patient B). Such a decision-level approach conflates the means with the ends. Instead, we elicit preferences directly over allocation outcomes to learn a utility function for policy optimization. We construct a novel preference elicitation algorithm for linear utilities that outperforms prior techniques in practice. Our algorithm has two phases. The first phase learns cutting planes through pairwise comparisons to rapidly shrink the space of possible attribute weights and warm-starts the second phase by eliminating dominated regions. The second phase then provably converges to the user's utility function. We apply our technique to heart transplant allocation, where a policy must balance competing objectives such as post-transplant outcomes, waitlist mortality, geographic ease, and equity. Using our algorithm, we conduct a user study to learn and aggregate a human-value-aligned utility function. We evaluate the linear representation against LLM-based methods supplied with users' natural language explanations, finding that none of these methods is more accurate than the learned linear utilities. We use the resulting utility function to optimize heart transplant policies that are significantly better aligned with human values. Compared to the hindsight optimum, the status quo policy achieves a competitive ratio of just 0.66, while our method is near-optimal with a competitive ratio of 0.95.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.28620 [cs.AI] |
| (or arXiv:2608.28620v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.28620 arXiv-issued DOI via DataCite |
Submission history
From: Itai Zilberstein [view email]
[v1]
Wed, 29 Jul 2026 13:33:10 UTC (208 KB)
[v2]
Fri, 9 Oct 2026 13:18:06 UTC (214 KB)
来源：arXiv：cs.AI · arxiv.org