TypoCast：以类型学条件化替代语言标签的多语言语音翻译前端

View PDF HTML (experimental)

Abstract:Multilingual speech translation frontends built on speech large language models typically condition on language identity, as a learned embedding or a prompted language name. Such a signal, however, is learned separately for each language and transfers nothing between typologically related languages. In this paper, we present TypoCast, a frontend that conditions on the typological profile of the source language rather than on its label alone. A typology-informed hierarchical language encoding represents each language by its morphological type, constituent order and family, so that supervision is shared among related languages. A dual-route mixture-of-experts adapter routes inputs through a shared typological route together with a per-language residual, and a typology-aware prompt conveys the same profile to the decoder. Extensive experiments on CVSS-C demonstrate that TypoCast surpasses state-of-the-art speech-to-speech translation systems on French, German and Spanish to English, averaged over the three directions, and outperforms the identity-conditioned S2ST-Omni on every language of a seven-language inventory, with only 228M trainable parameters. Ablation studies further reveal that the conditioning signal acts during training and is dispensable at inference: a model trained without it is consistently worse, whereas removing it from a trained model changes BLEU by less than 0.2, and expert usage is consistent with the signal shaping how adapter capacity is divided among languages.

Submission history

From: Yu Pan [view email]
[v1] Fri, 15 May 2026 15:01:45 UTC (351 KB)
[v2] Sun, 11 Oct 2026 15:59:33 UTC (510 KB)