arXiv：cs.AI· Pei-lin Li, Qingle Liu, Junyang Feng, Siyu Li, Sunqi Fan, Xin-Sheng Chen, Shuojin Yang·· 6 小时前
当上下文误导：大语言模型中的司法管辖式上下文学习
When Context Misleads: In-context Learning with Jurisdiction in Large Language Models
AI 导读
研究者提出 FakeContextBench 基准，覆盖七个领域的伪科学声明，用于评测 LLM 判断上下文是否应主导答案的能力。结果显示，仅靠大规模预训练不足以可靠辨别上下文权威，常见 ICL 微调方法反而使现实准确率相较基座模型最多下降 14.95 个百分点。
正文
Abstract:In-Context Learning (ICL) has become a cornerstone of modern LLM deployment. However, existing ICL post-training methods have a critical blind spot: they excel at extracting patterns from demonstrations while often neglecting context authority, the ability to determine whether contextual information should govern the final answer. To benchmark this capability, we introduce FakeContextBench, which contains pseudoscientific claims across seven domains. Our evaluation of commercial and open-source models shows that large-scale pre-training alone is insufficient for reliable context-authority discrimination. Moreover, prevalent ICL fine-tuning methods can increase susceptibility to misleading context, reducing reality accuracy by up to 14.95 percentage points relative to the base model. To address this trade-off, we propose Jurisdiction In-Context Learning (J-ICL), a post-training framework that incorporates context validation into the training objective. Across four model backbones, J-ICL improves ICLEval by an average of 5.84 percentage points and reality accuracy by 9.20 points over the corresponding base models. It also raises the Reality Rate by an average of 18.09 points relative to MetaICL and Symbol Tuning. These results demonstrate that ICL capability and resistance to deceptive context can be improved together. The benchmark is available at this https URL.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.27603 [cs.CL] |
| (or arXiv:2609.27603v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.27603 arXiv-issued DOI via DataCite |
Submission history
From: Pei-Lin Li [view email]
[v1]
Wed, 23 Sep 2026 09:21:14 UTC (2,053 KB)
[v2]
Mon, 12 Oct 2026 06:22:16 UTC (2,994 KB)
来源：arXiv：cs.AI · arxiv.org