arXiv：cs.CL· JiWoo Hwang, Lukas Edman, Alexander Fraser·· 10 小时前
多语言 BabyLM 预训练研究：语言曝光分布与顺序如何影响性能
Language Exposure Matters: Distribution and Order in Multilingual BabyLM Pre-training
AI 导读
一项多语言 BabyLM 预训练研究比较了英语、荷兰语和中文的曝光分布与顺序，发现混合训练在全部 15 组匹配配置中均优于课程学习式训练。部分语言主导的混合训练配置得分高于均衡配置，其中荷兰语主导配置整体表现最佳。
正文
Abstract:We investigate two aspects of multilingual pre-training with limited data, namely how language exposure is distributed across English, Dutch, and Chinese, and how that exposure is ordered during training. In particular, we compare balanced multilingual exposure to language-dominant exposure, where one language is exposed with more training data than the others. We also compare mixed training, where multilingual training data is used throughout training, with curriculum training, where monolingual exposure of the dominant language is followed by balanced multilingual exposure. Our results show that multilingual BabyLM performance varies with both the amount and order of language exposure. Mixed training outperforms curriculum learning across all 15 matched configurations, indicating that simultaneous multilingual exposure is more effective than staged exposure in our setting. Several language-dominant mixed training configurations achieve higher scores than the balanced configuration, suggesting that a balanced distribution is not necessarily the highest-performing configuration in our setting. Interestingly, the Dutch-primary configurations achieve the best overall performance.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.20102 [cs.CL] |
| (or arXiv:2610.20102v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20102 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: JiWoo Hwang [view email]
[v1]
Wed, 14 Oct 2026 12:37:19 UTC (492 KB)
来源：arXiv：cs.CL · arxiv.org