arXiv：cs.CL· Zahraa Al Sahili, Ioannis Patras, Matthew Purver·· 10 小时前
用推理引导翻译扩展多语言安全与推理评测
Scaling Multilingual Safety and Reasoning Evaluation through Reasoning-Guided Translation
AI 导读
研究提出推理引导翻译框架，将翻译策略选择与大规模生成分离，结合推理引导的配置选择、任务感知验证、针对性修复和独立质量筛查，构建覆盖 7 个安全、对话与数学推理资源、11 种语言的对齐评测集，两个需修复子集达到 99.94% 的条目-语言对覆盖率。对开源 LLM 的评测显示存在语言相关漏洞，包括拒绝行为不一致和低资源语言推理能力显著下降。
正文
Abstract:Large language models (LLMs) are increasingly deployed across languages, yet their safety and reasoning capabilities remain predominantly evaluated in English. Translating existing benchmarks offers a scalable approach to multilingual evaluation, but preserving task-specific meaning across languages presents significant challenges, particularly for harmful requests, mathematical reasoning, and conversational interactions. We introduce a reasoning-guided translation framework for constructing multilingual evaluations designed to preserve the structure and intended meaning of diverse benchmark tasks. Our approach separates translation strategy selection from large-scale generation, combining reasoning-guided configuration selection with task-aware validation, targeted repair, and independent quality screening. Using this framework, we develop a source-aligned evaluation collection spanning seven safety, conversational, and mathematical reasoning resources across 11 languages. The resulting collection achieves 99.94\% coverage of intended item--language pairs in the two subsets requiring repair. Evaluations of open-source LLMs reveal distinct language-dependent vulnerabilities, including inconsistent refusal behaviour and substantial reasoning degradation in lower-resource languages. These findings highlight the importance of task-sensitive benchmark construction and systematic quality assessment for understanding multilingual model behaviour beyond English.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.20221 [cs.CL] |
| (or arXiv:2610.20221v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20221 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zahraa Al Sahili [view email]
[v1]
Wed, 14 Oct 2026 13:58:39 UTC (109 KB)
来源：arXiv：cs.CL · arxiv.org