arXiv：cs.AI· Luca Giordano, Simon Razniewski·· 23 小时前
超越提问：评估 LLM 的陈述性知识
Beyond Questions: Evaluating the Stated Knowledge of LLMs
AI 导读
研究提出"开放知识评估"新范式，不再依赖预设问题，而是用开放式提示词考察 LLM 自发表达的知识。团队据此构建 BeQu（Beyond Questions）基准，包含 10,000 个实体及用于陈述验证的参考语料库，并据此评估多款语言模型，分析推理投入、模型规模、提示词格式和知识领域的影响。数据与排行榜已在该工作的 GitHub 仓库发布。
正文
Abstract:Parametric knowledge in large language models (LLMs) is a cornerstone of their success, yet remains poorly understood. Existing knowledge benchmarks typically rely on predefined questions (e.g., "What is the birth date of M.L. King?"), evaluating only knowledge that benchmark designers explicitly choose to query, a problematic availability bias. In this paper, we introduce open knowledge evaluation, a new paradigm for benchmarking the stated knowledge of LLMs. Instead of asking narrow questions, it evaluates models on the knowledge they choose to surface in response to open-ended elicitation prompts (e.g., "Tell me everything you know about M.L. King"). This shifts the focus from predefined answer retrieval toward characterizing the knowledge models naturally express. We instantiate this paradigm with BeQu (Beyond Questions), a benchmark of 10,000 entities paired with reference corpora for statement verification. Using BeQu, we evaluate a broad range of language models and analyze the effects of reasoning effort, model scale, prompt format, and knowledge domain. Data and leaderboard are available on this work's GitHub repository and at ANONYMIZED.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.26937 [cs.CL] |
| (or arXiv:2605.26937v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.26937 arXiv-issued DOI via DataCite |
Submission history
From: Luca Giordano [view email]
[v1]
Tue, 26 May 2026 12:29:18 UTC (3,278 KB)
[v2]
Mon, 10 Aug 2026 09:35:46 UTC (3,299 KB)
[v3]
Tue, 13 Oct 2026 09:41:02 UTC (3,227 KB)
来源：arXiv：cs.AI · arxiv.org