arXiv：cs.LG· Ali Alavi, Donald S. Williamson·· 3 小时前
你的语音质量指标有多容易被攻破？一套修正协议、一个基准及打补丁的收益
How Hackable Is Your Speech Quality Metric? A Corrected Protocol, a Benchmark, and What Patching Buys
AI 导读
针对语音质量预测器作为奖励时缺乏统一可攻破性度量的问题，研究者提出修正协议：以未扰动的往返输出而非原始输入作为参照，测得某防御的可攻破性从0.31降至0.08；并用多个随机种子训练的五个攻击者（成功率0.00至0.38）取最坏情况。
正文
Abstract:Speech quality predictors are increasingly used as rewards, yet no agreed measure of their hackability exists. The usual measurement has two flaws. First, the perturbation reaches the predictor through a processing chain -- here a neural codec -- that shifts the score on its own, which scoring against the raw input charges to the attack. Referencing the unperturbed round trip instead changes measured hackability by up to a factor of four (0.31 to 0.08 for one defence). Second, one trained attacker is a sample, not a measurement: five attackers differing only in random seed reach success rates from 0.00 to 0.38 against one fixed predictor, so a defence claim needs the worst case over several. Under this protocol, four published predictors differ widely: NISQA is hacked on 90% of utterances, SSL-MOS on 21%, DNSMOS on 14% and UTMOS on 6%. We then audit a closed attack-detect-patch loop. It hardens the predictor only in its own attack space, by less than the spread between attackers; a random-perturbation baseline matches it; and it costs up to 0.30 system SRCC out of domain. Enhancers post-trained against patched predictors hack them far less (PESQ -0.03 versus -0.23). Code, preregistration and run outputs are released.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.10899 [cs.LG] |
| (or arXiv:2610.10899v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10899 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ali Alavi [view email]
[v1]
Wed, 7 Oct 2026 20:55:58 UTC (113 KB)
来源：arXiv：cs.LG · arxiv.org