用方差感知评分奖励与 GRPO 提升 LLM 心脏医学问答能力

View PDF HTML (experimental)

Abstract:Large language models can support medical question answering, but privacy, cost, and reliability still limit their clinical use in focused domains. Can rubric-based reinforcement learning bring a locally deployable model close to much larger systems? We study Group Relative Policy Optimization (GRPO) for heart-focused medical question answering with rubric supervision from RaR-Medicine. Rather than rely on binary rubric aggregation or a single overall score, our variance-aware reward framework builds continuous rewards from criterion-level outcomes. It preserves partial credit and reduces reward sparsity. We adapt a Qwen3-14B policy with low-rank adapters and quantized weights. On a held-out heart-related HealthBench subset, the best GRPO variant improves accuracy from 0.362 to 0.502 and F1 from 0.532 to 0.668 over the base model, close to GPT-OSS-120B. An independent clinician rated most responses clinically acceptable (61%) and patient ready (63%), both above the automated accuracy; fewer than 2% were unsafe. Taken together, the results support rubric-based reinforcement learning as a practical route for focused medical language models.

Submission history

From: Yaser Banad [view email]
[v1] Fri, 17 Apr 2026 03:08:55 UTC (746 KB)
[v2] Tue, 13 Oct 2026 01:33:59 UTC (1,562 KB)