arXiv：cs.CL· Shuyuan Zheng, Yun-seo Lee, Xing Gao, Jialu Wu, Jiabao Hu, Katrina Qiyao Wang, Xinbo Lin, Qianying Liu, Chuan Xiao, Tao Huang·· 16 小时前
LLM 如何理解法哲学？LePhilBench 诊断大语言模型的法哲学推理能力
How Do LLMs Understand Legal Philosophy? Diagnosing Legal-Philosophical Reasoning in LLMs
AI 导读
研究者提出 LePhilBench，用于评测大语言模型的法哲学推理能力，包含两项基准任务与一个数据集。数据集测试集涵盖 220 份美国最高法院判决意见，由一位法学教授设计 14 个法哲学标注问题，三名受过法学训练并具备法理学正式培训的标注者为每份意见作答，共产生 9,240 个法哲学标签。两项任务分别考察 LLM 对法哲学的理解与其在法律推理中的应用。
正文
Abstract:Legal reasoning is a fundamental capability for large language models (LLMs) to complete legal AI tasks such as legal judgment prediction. In legal practice, this reasoning is shaped by legal philosophy: a judge's view of the nature and sources of law guides how legal materials are interpreted and applied, so judges working from the same statutes and precedents can reach different judgments. Existing research has largely overlooked whether LLMs can understand legal philosophy and apply it in legal reasoning, an ability we call \textit{legal-philosophical reasoning}. To fill this gap, we introduce LePhilBench, a benchmark for evaluating the legal-philosophical reasoning of LLMs that comprises two benchmarking tasks and a dataset. The two tasks evaluate how well LLMs understand legal philosophy and how well they apply it in legal reasoning, respectively. The dataset is manually and meticulously constructed. Its test set contains 220 U.S. Supreme Court opinions. A law professor designed fourteen annotation questions on legal philosophy, and three law-trained annotators with formal training in jurisprudence answered all fourteen questions for every opinion, yielding 9,240 legal-philosophical labels. LePhilBench takes the first step toward exploring legal-philosophical reasoning in LLMs.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.19427 [cs.CL] |
| (or arXiv:2610.19427v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19427 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shuyuan Zheng [view email]
[v1]
Wed, 14 Oct 2026 02:24:11 UTC (200 KB)
来源：arXiv：cs.CL · arxiv.org