arXiv：cs.AI· Runyuan Cai, Yu Lin, Yiming Wang, Xiaodong Zeng·· 18 小时前
Audio8-TTS：0.6B 参数零样本语音克隆模型实现 SOTA 自然度
Training a Compact, Strong, and Realistic Text-to-Speech Model
AI 导读
Audio8-TTS 是一个 0.6B 参数的零样本语音克隆模型，在清晰度和说话人相似度上达到 SOTA，感知自然度表现尤为突出。研究构建了基于开源 TTS 系统的质量阶梯，训练 realism critic 对照同一句子的真人录音评估生成语音，并用身份擦除目标防止其依赖生成器指纹。
正文
Abstract:Text-to-speech (TTS) systems are conventionally optimized for intelligibility and speaker similarity. However, neither objective captures the prosodic detail that distinguishes human performances from synthetic speech, leaving a realism gap that larger models and higher-quality data can narrow but not close. In the absence of an objective that directly targets this gap, realism remains a persistent challenge in TTS. We present Audio8-TTS, a compact 0.6B-parameter model for zero-shot voice cloning that achieves state-of-the-art performance in intelligibility and speaker similarity, with particularly strong results in perceptual naturalness. To align the training signal with human perception rather than signal-level fidelity, we construct a quality ladder from widely used open-source TTS systems and train a realism critic to assess generated speech against matched human recordings of the same sentences. An identity-erasure objective discourages the critic from relying on generator fingerprints. We train the model in three stages: multilingual supervised fine-tuning; group relative policy optimization (GRPO) with word error rate (WER) and speaker similarity (SIM) rewards; and GRPO guided by the realism critic. The final stage improves perceived prosodic naturalness and speaker similarity while preserving WER performance.
| Subjects: | Sound (cs.SD); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.14776 [cs.SD] |
| (or arXiv:2610.14776v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14776 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yiming Wang [view email]
[v1]
Sat, 10 Oct 2026 09:59:46 UTC (221 KB)
来源：arXiv：cs.AI · arxiv.org