arXiv：cs.CL· Junhyeok Lee, Han Jang, Kyu Sung Choi·· 3 小时前
通过路由重加权对 MoE 语言模型刻板印象的控制效果有限
Limited Stereotype Control Through Routing Reweighting in MoE Language Models
AI 导读
研究提出 Fairness-Aware Routing Equilibrium（FARE）诊断框架，结合人口统计路由画像、经验层选择和固定推理时重加权，评估了五种 MoE 架构。在选定操作点上，CrowS-Pairs 偏好变化至多 1.3 个百分点，DeepSeekMoE 不选择干预；唯一名义显著变化（Qwen1.5，p = 0.015）未通过多重比较校正，所测重加权程序对刻板印象的控制效果有限。
正文
Abstract:Demographic prompts are routed differently from neutral prompts in Mixture-of-Experts (MoE) language models, motivating tests of routing-level stereotype control. We introduce Fairness-Aware Routing Equilibrium (FARE), a diagnostic framework combining demographic routing profiles, empirical layer selection, and fixed inference-time reweighting, and evaluate five MoE architectures in English. At the selected operating points, CrowS-Pairs preference changes by at most 1.3 percentage points; DeepSeekMoE selects no intervention. Paired 95% confidence intervals exclude decreases larger than 2.2 points on each intervened model, and the only nominally significant change (Qwen1.5, p = 0.015) does not survive multiple-comparison correction. OLMoE and Qwen3 nevertheless change nearly every top-k expert set. Noise controls, random and truncated synthetic profiles, and hard masking also move preference by at most 1.5 points. Four generation protocols on OLMoE, Mixtral, Qwen1.5, and Qwen3 show no consistent change in the measured toxicity, lexical, or reference-overlap metrics. Selection and evaluation items overlap, so these comparisons are not independent evaluations. The tested reweighting procedure offers limited stereotype control.
| Comments: | 15 pages, 4 figures, 12 tables |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2603.27141 [cs.CL] |
| (or arXiv:2603.27141v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2603.27141 arXiv-issued DOI via DataCite |
Submission history
From: Junhyeok Lee [view email]
[v1]
Sat, 28 Mar 2026 05:33:42 UTC (493 KB)
[v2]
Thu, 8 Oct 2026 07:58:19 UTC (213 KB)
来源：arXiv：cs.CL · arxiv.org