语言模型 surprisal 对中文阅读时间预测能力的系统分析

View PDF HTML (experimental)

Abstract:This study analyzes the predictive power of LM-derived, token-level surprisal on Mandarin Chinese reading times. We first propose the Shortest Matching Sequence (SMS), an alignment scheme that maps between the word segmentation assumed by eye-tracking corpora and the LMs' subword tokenization, as the two tokenizations often disagree in the context of Mandarin Chinese. Then, using a suite of Chinese-Pythia models (14M-1.4B) trained on scratch with 30B tokens, we examine how well surprisal predicts first fixation duration, gaze duration, and total reading time in three paragraph-level eye-tracking corpora of Mandarin Chinese (GECO-CN, HKP, and MECO). Contrary to previous null findings, our results show that surprisal is predictive of Chinese reading times. However, whether predictive power scales with model size and the amount of training is corpus-specific: bigger models predict better in GECO-CN, whereas inverse scaling emerges in HKP and, at the largest sizes, in MECO. Subsequently, we tested one possible explanation for the inverse scaling in HKP and found that checkpoints whose surprisal remains closer to $n$-gram statistics are better predictors of reading. All in all, the predictive power of surprisal on Chinese reading time measurements is corpus-specific, which cautions against drawing scaling conclusions from a single corpus.

Comments:	15 pages, 3 figures
Subjects:	Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.04898 [cs.CL]
	(or arXiv:2610.04898v2 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.04898 arXiv-issued DOI via DataCite

Submission history

From: Hongao Zhu [view email]
[v1] Sun, 4 Oct 2026 03:19:26 UTC (5,872 KB)
[v2] Tue, 6 Oct 2026 06:56:02 UTC (5,872 KB)