arXiv：cs.CL· Ilia Kuznetsov, Rohan Nayak, Alla Rozovskaya, Iryna Gurevych·· 3 小时前
同行评审真的在衰退吗？跨会议与时间的评审质量分析
Is Peer Review Really in Decline? Analyzing Review Quality across Venues and Time
AI 导读
一项发表于 EMNLP-2026 的研究提出基于证据的评审质量比较框架，并应用于 ICLR、NeurIPS 和 *ACL，通过多维度量化方案结合 LLM 与轻量级测量评估评审质量。跨时间分析显示，各会议和年份的评审质量中位数并未出现一致的下降，与"评审质量衰退"的普遍说法相矛盾。
正文
Abstract:Peer review is at the heart of modern science. As submission numbers rise and research communities grow, the decline in review quality is a popular narrative and a common concern. Yet, is it true? Review quality is difficult to measure, and the ongoing evolution of reviewing practices makes it hard to compare reviews across venues and time. To address this, we introduce a new framework for evidence-based comparative study of review quality and apply it to major AI and machine learning conferences: ICLR, NeurIPS and *ACL. We document the diversity of review formats and introduce a new approach to review standardization. We propose a multi-dimensional schema for quantifying review quality as utility to editors and authors, coupled with both LLM-based and lightweight measurements. We study the relationships between measurements of review quality, and its evolution over time. Contradicting the popular narrative, our cross-temporal analysis reveals no consistent decline in median review quality across venues and years. We propose alternative explanations, and outline recommendations to facilitate future empirical studies of review quality.
| Comments: | To appear in EMNLP-2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2601.15172 [cs.CL] |
| (or arXiv:2601.15172v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2601.15172 arXiv-issued DOI via DataCite |
Submission history
From: Rohan Kumar Nayak [view email]
[v1]
Wed, 21 Jan 2026 16:48:29 UTC (2,246 KB)
[v2]
Thu, 8 Oct 2026 11:50:57 UTC (2,394 KB)
来源：arXiv：cs.CL · arxiv.org