arXiv：cs.AI· Alak Raad (Hamad Bin Khalifa University, Qatar), Noreldin Morssy (Hamad Bin Khalifa University, Qatar), Zeerak Suhail (Hamad Bin Khalifa University, Qatar), Serin Atiani (Princess Sumaya University for Technology, Jordan), Mustafa Jarrar (Hamad Bin Khalifa University, Qatar)·· 6 小时前
UDHRBench：面向大语言模型的人权基准测试
UDHRBench: Benchmarking Human Rights in Large Language Models
AI 导读
研究者推出 UDHRBench，用覆盖《世界人权宣言》全部 30 条条款的 1,124 道人权问题，对 ChatGPT、Grok、Gemini 和 Fanar 四个大语言模型进行评测，共获得 4,496 条由专家人工标注的回复，标注者间一致性 κ=0.801。
正文
Abstract:Large language models are increasingly deployed in contexts requiring adherence to ethical and legal norms, yet their alignment with internationally recognized human rights standards remains underexplored. We introduce UDHRBench, a large-scale benchmark for operationalising the evaluation of large language model compliance with the Universal Declaration of Human Rights. The dataset comprises 1,124 human rights confronting questions covering all 30 articles and 4,496 responses from four large language models (ChatGPT, Grok, Gemini, and Fanar), manually annotated by experts for alignment, with high inter-annotator agreement ($\kappa = 0.801$). Our analysis reveals systematic and non-trivial misalignments, showing that violations typically arise through seemingly rational reasoning: appeals to efficiency, risk management, or scientific neutrality, rather than explicit rejection of rights. We further propose an LLM-as-a-Judge framework for automated evaluation using zero-shot and few-shot prompting, achieving 89.8\% weighted F1-score in detecting human-rights misalignment, alongside a novel Alignment Leniency Score that quantifies how judge models resolve ambiguous cases. The dataset, annotations, and code will be released as open-source resources (this https URL).
| Comments: | Alak Raad and Noreldin Morssy contributed equally to this work |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.7, K.4.1 |
| Cite as: | arXiv:2610.15179 [cs.CL] |
| (or arXiv:2610.15179v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15179 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Serin Atiani [view email]
[v1]
Sat, 10 Oct 2026 20:59:28 UTC (270 KB)
来源：arXiv：cs.AI · arxiv.org