arXiv：cs.CL· Jiahong Qian·· 4 小时前
从不可靠到可用：分解 LLM 实体抽取的失败模式与置信信号——来自网络威胁情报的证据
From Unreliable to Usable: Decomposing Failure Modes and Confidence Signals in LLM Entity Extraction - Evidence from Cyber Threat Intelligence
AI 导读
一项预注册冻结评测用 DeepSeek-V4-Flash、GLM-5.3-Flash 及旗舰级对照 GPT-6.1-Sol 在 354 个事件活动语料上测试 LLM 实体抽取，micro precision 仅 0.03-0.34，置信校准误差 ECE 为 0.53-0.94，旗舰层级并未缩小差距。
正文
Abstract:Audits of large language model (LLM) extraction from full-length documents report severe unreliability, leaving two questions open: what the failures actually consist of, and whether confidence signals can turn unreliable output into usable output. We run a preregistered, fully frozen evaluation of two new-generation low-cost LLMs (DeepSeek-V4-Flash, GLM-5.3-Flash) with a flagship-tier control (GPT-6.1-Sol) on a public corpus of 354 incident campaigns (315 unique reports, 630 model-report evaluations), scoring extraction quality against gold, confidence calibration, and robustness to single-sentence perturbations. Three findings emerge. (1) Unreliability persists across tiers: micro precision is 0.03-0.34 and verbalized calibration error (ECE) 0.53-0.94 across both splits; the gap does not close at the flagship tier. (2) The dominant failure is not hallucination: a full mention census plus a three-tier annotation of 360 sampled candidates (two independent LLM annotators from vendors outside the evaluated set, Cohen's kappa = 0.79 on the preregistered acceptance line, fully disclosed) back-projects to 65.6% background mentions and 19.9% gold-scope gaps, with only 1.2% unsupported candidates. (3) Confidence ranks but does not verify: zero-cost self-consistency-frequency tiering passes a preregistered top-30% precision-gain bar in all six test-split model-type cells (4/6 on train), the zero-cost text-frequency x SC frequency combination wins 3, ties 2, loses 1 across cells, yet relation-level counterfactuals on all 45 targets show that a single misattribution sentence flips high-confidence judgments in 95.6-100% of cases, and even a background sentence explicitly marked as unrelated still binds a fake vulnerability in up to 26.7% of them. We conclude with a deployment boundary: confidence tiering may set review priorities, never truth.
| Comments: | 13 pages: 7-page main text within the 8-page limit, plus Limitations, references, and appendices A-J. Code, protocol, and frozen evaluation data will be made public |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.20417 [cs.CL] |
| (or arXiv:2610.20417v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20417 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiahong Qian [view email]
[v1]
Wed, 14 Oct 2026 16:13:09 UTC (460 KB)
来源：arXiv：cs.CL · arxiv.org