arXiv：cs.CL· Tzu-Ling Lin, Dong-Ting Yao, Teng-Fang Hsiao, Wei-Chih Chen, Hong-Han Shuai·· 4 小时前AI 评分41
HalluPeer：面向科学同行评审幻觉检测的分类学驱动基准
HalluPeer: A Taxonomy-driven Benchmark for Detecting Hallucinations in Scientific Peer Reviews
AI 导读
HalluPeer 是一个用于检测科学同行评审中幻觉的基准，提供论文内容、人工评审与注入幻觉评审对齐的三元组，支持检测、分类与定位标注。实验覆盖 12K 篇论文和 38K 条评审，显示现有检测器难以区分幻觉与合理批评；对真实评审的评估表明 HalluPeer 定义的幻觉模式确实存在于真实评审中。该工作已被 EMNLP Findings 2026 接收。
正文
Abstract:The growing scale of academic peer review has motivated the use of Large Language Models (LLMs) as review assistants, yet LLMs can generate fluent but unsupported claims that undermine review reliability. Existing hallucination benchmarks are not designed for peer review, where verification requires grounding claims in long, technical papers. We introduce HalluPeer, a benchmark for detecting hallucinations in scientific peer reviews, providing aligned triples of paper content, human-written reviews, and hallucination-injected reviews, annotated for detection, classification, and localization. Our pipeline induces a peer-review-specific hallucination taxonomy, identifies review contexts, and injects hallucinations with automated filtering. Experiments on 12K papers and 38K reviews show that existing detectors struggle to separate hallucinations from legitimate critique, while evaluation on authentic reviews demonstrates that HalluPeer-defined hallucination patterns occur in real peer reviews, highlighting the critical need for source-aware verification. Our project page can be found in this https URL
| Comments: | Accepted to EMNLP Findings 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.03580 [cs.AI] |
| (or arXiv:2609.03580v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.03580 arXiv-issued DOI via DataCite |
Submission history
From: Tzu-Ling Lin [view email]
[v1]
Thu, 3 Sep 2026 09:27:54 UTC (5,226 KB)
[v2]
Wed, 7 Oct 2026 09:42:16 UTC (5,112 KB)
来源：arXiv：cs.CL · arxiv.org