arXiv：cs.LG（机器学习，全量分类）· Sachin Gupta·· 14 小时前AI 评分41
VERITYGATE：面向结构化证据的 LLM 叙述四门校验框架与配对基准
VERITYGATE: A Four-Gate Schema-Level Faithfulness Framework and Paired Benchmark for Grounded LLM Narrations over Structured Evidence
AI 导读
VERITYGATE 是一个四门校验器，针对声明中的证据 ID、实体、数字和声明类型做 schema 级检查，并配套发布配对基准。在 r=0 和 r=1 下，用 GPT-4o-mini、Llama-3.3-70B 和 Claude Sonnet 4.6 各测 900 个实例，修复前 mini 有 80.3%、Sonnet 有 47.9% 的声明被拒绝。
正文
Abstract:Fluent LLM explanations may not follow the evidence from a structured system. We present VERITYGATE, a four-gate checker for declared evidence IDs, entities, numbers, and claim types. It checks a fixed schema; it does not verify every fact in the prose. At r=0 and r=1, we test 900 instances per setting (450 grounded-ungrounded pairs) with GPT-4o-mini, Llama-3.3-70B, and Claude Sonnet 4.6. Under this schema-level contract and before repair, 80.3% of mini claims and 47.9% of Sonnet claims fail. These are verifier rejection rates, not prose-hallucination rates. One repair pass raises claim survival from 19.7% to 28.0% for mini and from 52.1% to 54.3% for Sonnet. Verified claims per example change by +0.14 for mini, -0.71 for Llama, and -0.47 for Sonnet, so survival and output volume must be reported together. A second Sonnet pass gives no clear gain. At r=1, Gate 4 covers 97.0%, 98.7%, and 100% of failing claims for mini, Llama, and Sonnet. Small human studies support the rules but show gaps between schema checks and correct prose. A domain-specific GPT-4o judge test shows an order effect, so it is only a usefulness check. We release the code and data.
| Comments: | 16 pages, 5 figures, 7 tables. Accepted at Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026), co-located with EMNLP 2026. Code: this https URL Supplementary artifact: this https URL |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.00833 [cs.CL] |
| (or arXiv:2610.00833v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00833 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sachin Gupta [view email]
[v1]
Wed, 30 Sep 2026 23:44:51 UTC (47 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org