arXiv：cs.CL· Won Seok Jang, Youngbin Jang, Chanyoung Ko, Hyunwook Kim, Zonghai Yao, Hong Yu·· 4 小时前
DischargeBench：基于患者人格模拟的 LLM 出院教育者部署前评估基准
DischargeBench: Pre-Deployment Evaluation of LLM Discharge Educators through Persona-Grounded Patient Simulation
AI 导读
研究者推出 DischargeBench，一个让候选 LLM 教育者在开放式多轮对话中向人格化模拟患者进行出院教育的部署前基准，并构建含 16 个 ICD 章节、320 个病例的 MIMIC-IV-Ext-DischargeBench 数据集。
正文
Abstract:Effective discharge education is personalized: a clinician adapts the discharge plan to each patient's literacy, recall, and personality. Before large language models (LLMs) take on this role with real patients, they need to be evaluated in conversations with diverse patients, yet existing evaluations target static, artifact-generation, or question-answering tasks. We introduce **DischargeBench**, a pre-deployment benchmark in which a candidate LLM educator teaches a persona-grounded simulated patient in an open-ended, multi-turn conversation. We curate **MIMIC-IV-Ext-DischargeBench**, 320 cases across 16 ICD chapters, each paired with a patient persona that varies personality, education level, health literacy, and recall. Each conversation is scored on Conversation Quality, Topic Checklist Score (TCS), Simulated Comprehension Score (SCS), and a Claim-Level Safety Audit (CLSA) by an LLM judge validated against physician annotations. Across eight closed- and open-source LLMs, the best model scored only 0.70 on TCS and 0.72 on SCS, and distrustful and anxious patients lowered comprehension for every model. DischargeBench is intended as a screening step before human evaluation, not as evidence of readiness to educate patients.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.20827 [cs.CL] |
| (or arXiv:2609.20827v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.20827 arXiv-issued DOI via DataCite |
Submission history
From: Won Seok Jang [view email]
[v1]
Wed, 22 Jul 2026 13:24:24 UTC (5,705 KB)
[v2]
Tue, 13 Oct 2026 20:12:12 UTC (1,338 KB)
来源：arXiv：cs.CL · arxiv.org