arXiv：cs.AI· Li-Ching Chen, Andr\'es Franco Valiente, Yuta Ishiyama, Zachary R. Chalmers, William Chen, Kelsey Natsuhara, Taema Brinjikji, Madhumita Sushil, Travis Zack, Julian C. Hong·· 10 小时前
用部分可验证环境下的强化学习做结肠癌治疗方案规划
Reinforcement Learning In Partially Verifiable Environments for Colon Cancer Treatment Planning
AI 导读
研究者将肿瘤治疗方案规划建模为部分可观测下的交互式强化学习问题，用真实临床记录构建患者特异的过程感知奖励，并采用 GRPO 与 GDPO 优化 Llama-3.1-8B-Instruct。GDPO 训练的策略在 Mean@3 临床标量奖励上较基础 Llama 提升 79.60%，在同一交互环境中超过 GPT-5.2 达 18.82%，同时减少了训练中的查询次数。
正文
Abstract:Clinical decision-making is a sequential process of information collection, clinical inference, and forecasting under partial observability, often without a uniquely correct treatment decision. Large language models (LLMs) have demonstrated strong medical question-answering capabilities but extending these advances to clinical reasoning remains challenging because many clinical tasks are only partially verifiable. Recent clinical reinforcement learning (RL) work has therefore largely focused on diagnostic endpoints, where correctness can be more directly verified. Oncology treatment decision making is a particularly challenging instance of this problem that extends beyond medical question-answering in which multiple decision trajectories can be clinically defensible for the same partially observed patient state. We formulate oncology treatment planning as an interactive RL problem under partial observability and construct patient-specific, process-aware rewards from decision processes observed in real world clinical records. The agent must learn to sequentially acquire information from a structured patient representation before committing to a first-line treatment recommendation, and is evaluated on information gathering, treatment selection, and reasoning alignment. Using GRPO and GDPO to optimize Llama-3.1-8B-Instruct within this environment, RL substantially improves the base policy beyond frontier model capabilities. The GDPO-trained policy increases Mean@3 scalar clinical reward by $79.60\%$ over base Llama and exceeds GPT-5.2 by $18.82\%$ under the same interactive environment, while reducing query count during training. Overall, our results show that patient-specific, partially verifiable rewards can provide an effective learning signal for interactive clinical RL when both the patient state and the underlying clinical decision process are only partially observed.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.17426 [cs.LG] |
| (or arXiv:2610.17426v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17426 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Li-Ching Chen [view email]
[v1]
Mon, 12 Oct 2026 21:39:21 UTC (11,813 KB)
来源：arXiv：cs.AI · arxiv.org