arXiv：cs.CL· Neeraja Kirtane, Kuan-Hao Huang·· 6 小时前
语言向量引导多语言上下文学习：无需更新参数缩小跨语言差距
Language Steering for Multilingual In-Context Learning
AI 导读
研究者提出"语言向量"，通过计算平行语料在特定层的平均激活差，作为推理时的隐藏状态偏移量，在19种语言、三个任务和三个模型上无需更新参数即优于英语少样本提示。这些向量呈现可解释的语言结构：近亲语言聚成紧密簇，且能跨任务、跨语言迁移，近亲语言的向量恢复的增益更多。
正文
Abstract:If multilingual language model representations contain separable language-associated structure, then shifting between languages may require only simple activation-space interventions. To test this hypothesis, we study multilingual in-context learning (ICL) as a controlled setting: few-shot demonstrations are provided in English while the test query is in a target language, creating a clear cross-lingual transfer challenge. We propose language vectors, which are computed as mean activation differences between parallel source and target language examples at a particular layer and applied as inference-time offsets to the model's hidden states. Across three tasks spanning 19 languages and three models, our method improves over English few-shot prompting without any parameter updates. Beyond performance, the vectors encode interpretable linguistic structure: closely related languages form tight clusters, and vectors transfer effectively across tasks and languages, with closely related languages' vectors recovering more of the gain than distant ones. Together, these findings suggest that language occupies structured directions in multilingual activation space, and that inference-time interventions on English demonstrations can narrow the cross-lingual ICL gap.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2602.02326 [cs.CL] |
| (or arXiv:2602.02326v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2602.02326 arXiv-issued DOI via DataCite |
Submission history
From: Neeraja Kirtane [view email]
[v1]
Mon, 2 Feb 2026 16:52:09 UTC (734 KB)
[v2]
Wed, 1 Apr 2026 17:54:08 UTC (727 KB)
[v3]
Wed, 14 Oct 2026 15:10:55 UTC (7,157 KB)
来源：arXiv：cs.CL · arxiv.org