arXiv：cs.AI· Xinyang Liao, Lingyu Li, Huacan Liu, Tianle Gu, Yang Yao, Tong Zhu, Yan Teng, Yingchun Wang·· 21 小时前
PseudoBench：衡量智能体自动科研如何助推伪科学
PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience
AI 导读
PseudoBench 是一个对抗性基准，用覆盖五个领域的 200 组伪科学主张-证据对，通过从实验到写作的端到端流程评估智能体自动科研系统能否识别并抵制伪科学叙事。测试的七个前沿智能体几乎不拒绝，最高抵制率仅 27.4%，且能力更强的智能体会用更复杂的科学语言包装伪科学，反而提升其表面可信度。
正文
Abstract:As Large Language Model based agents enter autonomous scientific research, their ability to resist pseudoscience becomes increasingly important. Otherwise, such systems may rapidly generate plausible yet misleading studies that contaminate academic literature and erode trust in science. We present PseudoBench, an adversarial benchmark for evaluating whether agentic auto-research systems can identify and resist pseudoscientific narratives. PseudoBench contains 200 curated pseudoscientific claim-evidence pairs across five domains and evaluates agents through an end-to-end research pipeline from experiments to writing. Testing seven state-of-the-art agents, we find that current systems readily produce persuasive reports that align with pseudoscientific premises with near-zero refusal rates and the highest resistance of only 27.4%. Stronger agents risk packaging pseudoscience in more sophisticated scientific language, increasing its apparent credibility. These findings reveal an alarming capacity to fuel pseudoscience, calling for scientific alignment before widespread deployment.
| Comments: | 34 pages, 25 figures |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.18060 [cs.AI] |
| (or arXiv:2606.18060v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2606.18060 arXiv-issued DOI via DataCite |
Submission history
From: Xinyang Liao [view email]
[v1]
Tue, 16 Jun 2026 15:37:02 UTC (10,727 KB)
[v2]
Tue, 13 Oct 2026 08:26:30 UTC (8,711 KB)
来源：arXiv：cs.AI · arxiv.org