arXiv：cs.LG· Qihao Wen, Jiahao Wang, Yang Nan, Pengfei He, Ravi Tandon, Han Xu·· 7 小时前AI 评分34
通过嵌入向量扰动定位 LLM 推理中的不确定性
Uncertainty Localization in LLM Reasoning via Embedding Perturbations
AI 导读
一项研究提出用嵌入向量扰动来定位 LLM 推理轨迹中的不确定位置，发现不确定的中间续写更易出现在对前序 token 嵌入扰动高度敏感的 token 上。实验显示，这类基于扰动的指标在定位不确定中间步骤上优于基于概率、采样和贝叶斯的方法，且更简单高效。
正文
Abstract:Large Language Models (LLMs) have achieved significant breakthroughs across various domains, but they can still produce unreliable or misleading outputs. For responsible LLM applications, uncertainty quantification techniques are used to estimate a model's uncertainty about its outputs, indicating the likelihood that those outputs may be problematic. For LLM reasoning tasks, it is essential to estimate uncertainty not only in the final answer but also in the intermediate reasoning process, particularly to identify where uncertainty arises. Such information may enable more fine-grained and targeted interventions during inference. In this study, we investigate which metrics can effectively localize uncertain places within an LLM reasoning trajectory. Our study reveals that uncertain intermediate continuations are more likely to occur at tokens that are highly sensitive to perturbations in the embeddings of preceding tokens. In our experiments, we show that such perturbation-based metrics achieve stronger performance in localizing uncertain intermediate steps than baseline methods, including probability-based, sampling-based, and Bayesian-based approaches. Meanwhile, our proposed metrics also enjoy good simplicity and efficiency.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2602.02427 [cs.LG] |
| (or arXiv:2602.02427v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.02427 arXiv-issued DOI via DataCite |
Submission history
From: Qihao Wen [view email]
[v1]
Mon, 2 Feb 2026 18:27:26 UTC (1,172 KB)
[v2]
Wed, 13 May 2026 18:26:04 UTC (898 KB)
[v3]
Tue, 29 Sep 2026 05:07:17 UTC (1,215 KB)
[v4]
Mon, 5 Oct 2026 19:19:52 UTC (1,215 KB)
来源：arXiv：cs.LG · arxiv.org