arXiv：cs.AI· Bharath Kumar Bolla, Chaitali Deb Purkayastha, Vishnu Surya Reddy Nandi·· 5 小时前
临床决策支持中 LLM 的答案正确不代表推理可靠：准确率、一致性与共识是误导性指标
Right Answer, Wrong Reason: Accuracy, Consistency, and Consensus Are Misleading Indicators of LLM Faithfulness in Clinical Decision Support
AI 导读
研究提出 ESI、CFS、PSS 三个可解释指标衡量临床 LLM 的忠实性缺口，在 150 道 MedQA-USMLE 题（900 次模型-题目观测）上评测六个 LLM，发现仅 23.3% 被引用的临床概念是因果必要的。
正文
Abstract:Clinical Large Language Models (LLMs) achieve strong medical-exam accuracy; however, a correct answer does not guarantee that the explanation names the concepts that actually drove the decision. We introduce three lightweight, directly interpretable metrics for this faithfulness gap: the Explanation Stability Index (ESI), which measures reasoning consistency across repeated queries; the Causal Faithfulness Score (CFS), which tests whether cited concepts drive predictions via concept ablation; and the Perturbation Stability Score (PSS), which measures robustness to semantic-preserving paraphrases. By evaluating six LLMs on 150 MedQA-USMLE questions (900 model-question observations), we found that only 23.3% of the cited clinical concepts were causally necessary. Correct answers had lower CFS than incorrect answers (0.212 vs. 0.398), answer consistency negatively predicted CFS (Spearman r = -0.466), and model pairs could agree on answers while sharing only 8.8% of cited reasoning concepts. These results show that accuracy, consistency, and consensus are incomplete safety signals for clinical decision-making support. The evidence is behavioral rather than mechanistic: concept ablation tests counterfactual sensitivity of outputs, not internal circuits.
| Comments: | Accepted and received best paper award from ICETCI 2026 Conference, this https URL |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.32817 [cs.AI] |
| (or arXiv:2609.32817v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.32817 arXiv-issued DOI via DataCite |
Submission history
From: Vishnu Surya Reddy Nandi [view email]
[v1]
Sat, 26 Sep 2026 17:44:22 UTC (619 KB)
[v2]
Sat, 10 Oct 2026 19:26:58 UTC (619 KB)
来源：arXiv：cs.AI · arxiv.org