arXiv：cs.CL· Cameron Morin, Matti Marttinen Larsson·· 6 小时前
用 LLM 大规模自动化语料标注：英语 consider 构式的变异与演变
A large-scale pipeline for LLM-assisted corpus annotation: variation and change in the English consider construction
AI 导读
研究者提出一套四阶段 LLM 语料标注流水线（提示词工程、事前评估、批量自动处理、事后验证），通过 OpenAI API 在不到 60 小时内标注了 COHA 语料库中 143,933 条 consider 索引行，两项复杂标注任务准确率均达 98% 以上。
正文
Abstract:As natural language corpora expand at an unprecedented rate, manual annotation remains a significant methodological bottleneck in corpus linguistic work. We address this challenge by presenting a scalable pipeline for automating grammatical annotation in voluminous corpora using large language models (LLMs). Unlike previous supervised and iterative approaches, our method employs a four-phase workflow: prompt engineering, pre-hoc evaluation, automated batch processing, and post-hoc validation. We demonstrate the pipeline's accessibility and effectiveness through a diachronic case study of variation in the English evaluative consider construction (consider X as/to be/Ø Y). We annotate 143,933 'consider' concordance lines from the Corpus of Historical American English (COHA) via the OpenAI API in under 60 hours, achieving 98%+ accuracy on two sophisticated annotation procedures. A Bayesian multinomial GAM fitted to 44,527 true positives of the evaluative construction reveals previously undocumented genre-specific trajectories of change, enabling us to advance new hypotheses about the relationship between register formality and competing pressures of morphosyntactic reduction and enhancement. Our results suggest that LLMs can perform a range of data preparation tasks at scale with minimal human intervention, unlocking substantive research questions previously beyond practical reach, though implementation requires attention to costs, licensing, and other ethical considerations.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2510.12306 [cs.CL] |
| (or arXiv:2510.12306v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2510.12306 arXiv-issued DOI via DataCite |
Submission history
From: Cameron Morin [view email]
[v1]
Tue, 14 Oct 2025 09:06:14 UTC (1,103 KB)
[v2]
Tue, 10 Feb 2026 15:03:47 UTC (1,698 KB)
[v3]
Mon, 15 Jun 2026 11:06:51 UTC (1,856 KB)
[v4]
Tue, 13 Oct 2026 21:04:48 UTC (2,584 KB)
来源：arXiv：cs.CL · arxiv.org