arXiv：cs.CL· Ryan Cotterell·· 6 小时前AI 评分35
惊讶理论是同义反复：缺乏理性基础则不可证伪
Surprisal Theory is Tautological (without Rational Grounding)
AI 导读
EMNLP 2026 论文指出，惊讶理论主张语言单位处理难度是某语言模型下惊讶度的仿射函数，但这实为同义反复：只要技术条件温和，任何非负难度度量都能找到对应语言模型。二十年来心理语言学默认该模型为训练语料生成分布，近期实证工作却表明语料拟合更好的模型反而更差地预测处理难度。作者认为打破同义反复需引入理性主义约束，即语言模型须源自记忆限制或处理目标等非经验驱动的理解者模型。
正文
Abstract:Surprisal theory holds that the human processing difficulty of a linguistic unit in context is an affine function of its surprisal under some language model. I argue this claim is a tautology without further constraint: for any non-negative difficulty measure over units in context, there exists a language model whose surprisal is an affine function of it under mild technical conditions. Therefore, because any pattern of difficulty is consistent with some language model, without an additional constraint on the language model, surprisal theory makes no falsifiable predictions. The tautology was long obscured by an assumption implicit in two decades of psycholinguistic work---that the relevant language model is the distribution that generated the training corpus, so that improving corpus fit improves predictions of human behavior. Recent empirical work has undermined this assumption, demonstrating that better corpus models can be worse predictors of processing difficulty. I conclude that breaking the tautology requires a rationalist intervention, i.e., the relevant language model must be derived from a non-empirically motivated model of the comprehender, which could be based on, for instance, memory constraints or processing goals, and that, thus, does not depend on the behavioral data surprisal theory is meant to explain.
| Comments: | EMNLP 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.21574 [cs.CL] |
| (or arXiv:2607.21574v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.21574 arXiv-issued DOI via DataCite |
Submission history
From: Ryan Cotterell [view email]
[v1]
Thu, 23 Jul 2026 17:54:37 UTC (44 KB)
[v2]
Tue, 6 Oct 2026 19:31:53 UTC (46 KB)
来源：arXiv：cs.CL · arxiv.org