arXiv：cs.CL· Ming-Hao Hsu, Yuxiang Wang, Yingda Shen, Zhizheng Wu·· 10 小时前
AFFIRM：用冻结仿射场修复文本更新后的语音访问
Repairing Speech Access After Text Updates with Frozen Affine Fields
AI 导读
研究者提出 AFFIRM，通过冻结知识检查点、以可拆卸算子在一次前向中修复文本更新后的语音访问。在 SLUE-SQA-5 录音问题上，全规模 PopQA 拟合的算子无需重拟合即可迁移，ROUGE-L 比配对的无修复基线提升 0.0941。但在合成语音 PopQA 上，AFFIRM 相比 SALSA 全局引导和低秩 ReFT 未显现明确优势，表明文本习得与语音访问在更新后可能分离，需分别验证和修复。
正文
Abstract:Speech-capable language models inherit most of their knowledge from text-rich language backbones, and continued model maintenance is likewise dominated by text updates. Knowledge acquired through a text update can nevertheless remain difficult to access when the same query is spoken. We formulate this mismatch as post-update speech access repair, where the knowledge-bearing checkpoint stays frozen and a detachable operator repairs speech-only access in one model pass. We introduce Affine Fields for Interface Repair across Modalities (AFFIRM), which first learns and freezes a base correction for speech hidden states, then adds query-specific corrections with zero mean over the operator-training queries. Paired speech and text states provide the coordinates that condition these corrections on each query. On balanced synthetic-speech PopQA, paired comparisons do not establish a clear advantage for AFFIRM over global SALSA steering or low-rank ReFT interventions under the same frozen single-state constraint. On recorded SLUE-SQA-5 questions with text passages, a full-scale PopQA-fitted AFFIRM operator transfers without refitting and improves ROUGE-L by 0.0941 over its paired No Repair baseline. Low-rank ReFT interventions also transfer positively, whereas the matched SALSA variant falls 0.0707 below its paired baseline. These results show that text acquisition and speech access can diverge after an update and should be verified and repaired separately.
| Subjects: | Computation and Language (cs.CL); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2610.20253 [cs.CL] |
| (or arXiv:2610.20253v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20253 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ming-Hao Hsu [view email]
[v1]
Wed, 14 Oct 2026 14:24:38 UTC (52 KB)
来源：arXiv：cs.CL · arxiv.org