arXiv：cs.CL· Khoi-Nguyen Tran·· 11 小时前
perturber：面向语义保持文本扰动的无状态服务
perturber: A Stateless Service for Semantics-Preserving Text Perturbations
AI 导读
perturber 是一个无状态服务和 Python 库，每次调用只施加一个语义保持扰动，扰动来自覆盖 11 个家族的 68 种自研扰动目录。服务与库由同一原语构建，调用方通过链式调用来组合，因此两者输出完全一致。该工作评估工具本身的正确性、可复现性与成本，而非任何模型的鲁棒性，并逐扰动测量语义保持程度。
正文
Abstract:Large language model (LLM) evaluations on clean benchmarks overstate model robustness: real inputs arrive perturbed, and these meaning-preserving variants routinely move model answers. Studying this systematically requires applying perturbations one at a time, reproducibly, and identically wherever they run, yet existing tools couple them to a benchmarking stack rather than packaging them as a reusable transform layer. We present perturber, a stateless service and Python library that applies exactly one semantics-preserving perturbation per call, drawn from a first-party catalog of 68 perturbations across 11 families. It turns on a single primitive, with composition left to the caller chaining calls, so the service and the library produce identical output by construction. We evaluate the tool itself, its correctness, reproducibility, and cost, rather than any model's robustness, measuring meaning-preservation per perturbation rather than assuming it. The demonstration adds a live playground, an introspectable catalog for benchmark selection, and a one-command reproducible evaluation.
| Comments: | 14 pages (6 main + references and appendix), 5 figures, 13 tables. Code: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.12667 [cs.CL] |
| (or arXiv:2610.12667v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.12667 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Khoi-Nguyen Tran [view email]
[v1]
Thu, 8 Oct 2026 19:44:05 UTC (153 KB)
来源：arXiv：cs.CL · arxiv.org