arXiv：cs.CL· Zixiang Xu, Zirui Song, Chiyu Zhang, Xiuying Chen, Xi Liu, Xiyang Hu, Yue Zhao·· 3 小时前
JevOut：自然上下文如何翻转决策模型的判断
JevOut: Natural Context Can Flip Decision Models
AI 导读
研究提出 JevOut，通过概率引导的上下文优化构造自然背景文本，在 Jev 等四个决策系统、七个数据集上，将 61.4%-73.2% 的原本正确决策重定向为预设错误选项。这些文本不包含显式选答指令，且能产生高置信度错误并跨模型迁移。盲评中 250 个成功上下文有 91.6% 被三名独立标注者多数判定为自然且不改变答案。
正文
Abstract:An ordinary-looking background detail can turn a correct model decision into a confident mistake. We demonstrate this fragility in four decision systems, including Jev, across seven datasets covering knowledge, reasoning, and tool routing. Within 64 accepted target evaluations per decision, we uncover short context additions that redirect 61.4%-73.2% of each system's initially correct decisions toward a wrong option fixed in advance. The additions supply background or procedural information rather than explicit answer-selection instructions, leaving the original question and choices intact. We construct them through probability-guided context optimization, which uses shifts in the option distribution to refine surrounding text under naturalness and answer-preservation constraints. Redirection affects initially confident decisions, often produces high-confidence wrong choices, and transfers across models. In blinded human evaluation, 91.6% of 250 sampled successful contexts are judged natural, answer-preserving, and free of decisive answer-changing evidence by a majority of three independent annotators. These findings expose a weakness in current decision models: context that looks entirely compatible with an input can redirect the choices that agents, routers, and evaluators rely on.
| Comments: | 46 pages, 8 figures, 30 tables. V2: substantially strengthened empirical evaluation with blinded human validation, a budget-matched independent-generation baseline, and repeatability analyses; expanded related work and updated author list. Homepage: this https URL ; Code: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.30243 [cs.CL] |
| (or arXiv:2609.30243v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.30243 arXiv-issued DOI via DataCite |
Submission history
From: Zixiang Xu [view email]
[v1]
Thu, 24 Sep 2026 17:57:07 UTC (493 KB)
[v2]
Thu, 8 Oct 2026 08:43:50 UTC (698 KB)
来源：arXiv：cs.CL · arxiv.org