arXiv：cs.CL· Santhosh Kumar Kasa, Siva Rajesh Kasa, Sumit Negi·· 3 小时前AI 评分46
相同文本不同预测：文本分类器中的服务上下文非确定性
Same Text, Different Prediction: Serving-Context Nondeterminism in Text Classifiers
AI 导读
一项系统性研究训练了 180 个模型，涵盖判别式、伪生成式和完全生成式分类器，在固定 checkpoint 和文本的情况下评估四类服务上下文的影响。仅改变 batch shape 在 fp32 下不改变任何标签，但在 bf16 下可使预测概率质量移动高达 56.7 个百分点，标签变化集中在 margin 较小的样本上。
正文
Abstract:Deterministic inference is essential for reliable and trustworthy machine learning. Prior studies of text generation have shown that changing factors such as batch size, batch composition, hardware, or inference engine can alter the generated text, even when the prompt, model parameters, and sampling randomness are fixed. These differences have been attributed in part to floating-point non-associativity, shape-dependent kernel selection, and other implementation-level differences in numerical execution. However, it remains unclear whether, when, and to what extent the same factors affect text classification. We present a systematic study of serving-context non-invariance in text classifiers, which prior work has measured only through generated text. We train 180 models spanning discriminative, pseudo-generative, and fully generative classifier formulations and evaluate each across four categories of serving contexts, holding the checkpoint and the text fixed. Label stability does not imply score stability. Changing only the batch shape changes no labels across fp32 comparisons, yet under bf16 it moves up to 56.7 percentage points of predicted probability mass, with label changes concentrated at small margins. Fully generative classifiers change more labels than their discriminative counterparts under the same serving changes. We derive sufficient conditions for label stability under each serving change and give a separate mitigation for each mechanism. Our results identify and quantify the serving conditions that must be fixed for reproducible text classification.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.09111 [cs.CL] |
| (or arXiv:2610.09111v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09111 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Siva Rajesh Kasa [view email]
[v1]
Tue, 6 Oct 2026 21:02:25 UTC (283 KB)
来源：arXiv：cs.CL · arxiv.org