arXiv：cs.LG（机器学习，全量分类）· Negin Ashrafi, Jia Luo, Stacey M. Frumm, Roxana Daneshjou·· 7 小时前AI 评分46
OpenMTB-Audit：LLM 分子肿瘤委员会安全评估中的过度拒答与临床专家视角
OpenMTB-Audit: Exposing Over-Refusal and Clinical Expert Perspectives in LLM-Based Molecular Tumor Board Safety Evaluation
AI 导读
研究者发布开源基准 OpenMTB-Audit，含 500 个合成非小细胞肺癌病例、五类对抗性错误和四种安全标签。八种 LLM 配置在 83.3-100% 的真实 Partially Supported 病例中丢失该标签，靠标签坍缩而非临床校准推理获得高安全分。配套的 MTB-AuditAgent 七模块框架将过度拒答降至 6.7%，准确率达 91.2%（95% CI：88.6-93.6%）。
正文
Abstract:Molecular tumor boards integrate genomic findings, clinical context, and therapeutic evidence to support precision oncology. As AI enters this workflow, a key safety challenge is distinguishing truly unsupported recommendations from evidence-supported options that still require oncologist review because of incomplete information, poor ECOG performance status, or other clinical caveats. We introduce OpenMTB-Audit, an open-source benchmark of 500 synthetic non-small cell lung cancer cases spanning five adversarial error categories and four safety labels: Supported, Partially Supported, Unsupported, and Insufficient Information. Across eight large language model configurations, we identify pervasive over-refusal: all LLM configurations failed to retain the Partially Supported label in 83.3-100% of true Partially Supported cases, achieving high aggregate safety scores through label collapse rather than clinically calibrated reasoning. To address this limitation, we developed MTB-AuditAgent, a deterministic seven-module framework separating evidence verification, missing-information detection, safety classification, and abstention. It reduces over-refusal to 6.7% and achieves 91.2% accuracy (95% CI: 88.6-93.6%). A two-oncologist annotation study found disagreement concentrated at the boundary between information sufficiency and treatment optimization, underscoring the need to preserve clinically meaningful distinctions.
| Comments: | Accepted for oral presentation and publication at the Pacific Symposium on Biocomputing (PSB) 2027 |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.01497 [cs.AI] |
| (or arXiv:2610.01497v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.01497 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Negin Ashrafi [view email]
[v1]
Thu, 1 Oct 2026 11:39:31 UTC (9,224 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org