arXiv：cs.CL· Anna Mosolova, Djam\'e Seddah·· 3 小时前AI 评分44
多语言 LLM 的日常知识盲区：TriviaRoomQA 基准揭示流行文化短板
When Trivia Is Not Trivial: Everyday Knowledge Failures in Multilingual LLMs
AI 导读
研究者推出多语言基准 TriviaRoomQA，用 288 个主题的问答评估 LLM 的日常与长尾知识，含 6 种欧洲语言的 3300 道平行选择题及 5340 道法语专属题。对 30 个 7B 至 70B 开源权重模型的测试显示，模型在历史、地理、数学等知识密集型主题上表现强劲，但在明星、音乐、电影、新闻等日常流行文化主题上明显较弱，且同一问题在不同语言下表现不一。
正文
Abstract:Quiz rooms, trivia nights, and quiz shows challenge human knowledge across a wide range of topics, from canonical facts to everyday culture. In this paper, we examine whether large language models (LLMs) can perform competitively in such settings, using quiz-style questions to test them on both common and niche topics. We introduce TriviaRoomQA, a multilingual benchmark designed to evaluate everyday, culturally grounded, and long-tail knowledge across 288 topics. The benchmark contains 3,300 parallel multiple-choice questions in six European languages and additional 5,340 French-only questions for a more fine-grained case study. We evaluate 30 open-weight LLMs from European, Asian, and North American providers, covering models from 7 to 70B parameters. We find that models are strong on knowledge-intensive topics such as history, geography, and mathematics, but substantially weaker on everyday popular-culture topics such as celebrities, music, movies, and news. Moreover, model performance varies across languages even for the same underlying questions, suggesting that access to factual knowledge is not always language-independent. In sum, our dataset and experiments demonstrate an important knowledge gap which is not captured by existing academic-based saturated benchmarks.
| Comments: | EMNLP 2026 Findings |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.21445 [cs.CL] |
| (or arXiv:2607.21445v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.21445 arXiv-issued DOI via DataCite |
Submission history
From: Anna Mosolova [view email]
[v1]
Thu, 23 Jul 2026 15:52:00 UTC (41,704 KB)
[v2]
Wed, 7 Oct 2026 16:43:46 UTC (14,228 KB)
来源：arXiv：cs.CL · arxiv.org