arXiv：cs.CL· Jianan Pan, Yiwen Gu, Xinze Li, Rui Wang, Kejie Huang·· 6 小时前AI 评分33
EMODE：面向情感感知语音语言建模的动态副语义专家
EMODE: Dynamic Para-Semantic Experts for Emotion-Aware Speech Language Modeling
AI 导读
EMODE 是一个基于动态副语义专家（DPSE）的情感感知语音语言模型，DPSE 将连续语音特征分解为语义与副语言两条路径，动态路由并在融入语言模型前融合。该模型采用语义预热、副语言激活、联合精炼三阶段课程训练，并引入 OEG、SAA 和 GDR 三项引导机制。在 SER 测试、共情回复评估及新建双语 MEPA 基准上，EMODE 改善了词汇保真度与情感敏感度的平衡，并增强了基于情感的回复生成。
正文
Abstract:Large speech language models have demonstrated strong capabilities in unified cross-modal understanding and generation, yet paralinguistic cues, especially emotion, remain difficult to preserve. Existing systems typically rely on entangled acoustic representations, which allow the underlying language model to depend excessively on recovered lexical content instead of grounding its behavior in acoustic-prosodic evidence. We address this limitation with EMODE, an emotion-aware speech language model built around \textbf{Dynamic Para-Semantic Experts (DPSE)}. DPSE decomposes continuous speech features into semantic and paralinguistic pathways, routes them dynamically, and fuses them before integration into the language model. To turn this structural decomposition into functional specialization, EMODE is trained with a three-stage curriculum consisting of semantic warm-up, paralinguistic activation, and joint refinement, guided by Orthogonal Expert Guidance (OEG), Semantic-to-Acoustic Alignment (SAA), and Gating Diversity Regularization (GDR). Experiments on SER test, empathetic response evaluation, and the newly constructed bilingual MEPA benchmark show that EMODE improves the balance between lexical fidelity and emotional sensitivity, strengthens affect-grounded response generation, and exposes the value of explicit para-semantic factorization for robust cross-corpus emotion understanding.
| Subjects: | Computation and Language (cs.CL); Multimedia (cs.MM); Sound (cs.SD) |
| Cite as: | arXiv:2610.06956 [cs.CL] |
| (or arXiv:2610.06956v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.06956 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jianan Pan [view email]
[v1]
Sat, 3 Oct 2026 14:33:36 UTC (1,003 KB)
来源：arXiv：cs.CL · arxiv.org