arXiv：cs.CL· Arzu Burcu G\"uven, Anna Rogers, Leonie Weissweiler, Joakim Nivre, Rob van der Goot·· 13 小时前
BLiMPPI：33 种语言的极性词最小对立对评测基准
BLiMPPI: Benchmark of Linguistic Minimal Pairs for Polarity Items
AI 导读
BLiMPPI 是一个覆盖 33 种语言、超过 12,000 组极性词（NPI）最小对立对的多语言评测套件，通过 Universal Dependencies 树库自动发现 NPI 并标注其允准环境后扰动生成。
正文
Abstract:Negative polarity items (NPIs), such as "anymore" and "ever", are grammatical only in select environments, predominantly in the context of negation. Because their grammaticality depends on both syntax and semantics, they have a rich, multilingual literature in theoretical linguistics, yet work on NPIs in language models is almost entirely confined to English. We introduce BLiMPPI, a multilingual evaluation suite of over 12,000 NPI minimal pairs in 33 languages, built by a pipeline that discovers NPIs in Universal Dependencies (UD) treebanks, labels their licensing environments, and perturbs these into minimal pairs. Evaluating 12 multilingual and 2 monolingual models alongside n-gram baselines, we find that multilingual models struggle most with lower-resource languages and with pairs that are not n-gram solvable. We further examine the effects of NPI typology, language family, licensing variability, and NPI-licenser distance. Data: this https URL
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.13607 [cs.CL] |
| (or arXiv:2610.13607v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13607 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Arzu Burcu Güven [view email]
[v1]
Fri, 9 Oct 2026 13:55:00 UTC (286 KB)
来源：arXiv：cs.CL · arxiv.org