arXiv：cs.AI· Nishanth Nayakanti, Prasang Gupta, Ashutosh Bilthare, Kevin Paul·· 6 小时前AI 评分34
仅凭判定结果后训练的小模型能否恢复证据？拒绝采样与纯标签训练对比
Verdicts Without Annotated Evidence: Rejection Sampling or Label-Only Post-Training for Evidence Recovery?
AI 导读
在判定结果是唯一留存记录的审查流程中，研究测试了小语言模型仅用判定结果后训练、全程无人工证据标注时能恢复多少证据。在 ContractNLI 上，纯标签训练达到准确率 0.896、span F1 0.564，拒绝采样为 0.797 和 0.556。逐字引用率在纯标签训练下从 0.597 升至 0.729，拒绝采样下升至 0.701，两种方法都能在无标注情况下改善证据恢复。
正文
Abstract:In many review workflows the verdict is the only thing retained. The passages behind it are not marked, because that annotation costs far more than recording the decision. We measure how much of that evidence a small language model can recover when it is post-trained on the verdicts alone, with no human evidence labels at any stage. On ContractNLI the human evidence spans are held out until evaluation. Matching the recorded verdict and agreeing with those spans are not the same thing: across six systems the two scores are only weakly related and rank the systems differently, so accuracy is a poor guide when the citations have to be reviewable. Label-only training on the bare verdict reaches accuracy 0.896 and span F1 0.564. Rejection sampling, which keeps a generated trace only when its verdict matches the record and then picks one by an automatic source-grounding score, reaches 0.797 and 0.556, against 0.747 and 0.493 before training. Verbatim citation rises from 0.597 to 0.729 under label-only training and to 0.701 under rejection sampling. One seed on one corpus cannot say which method is better, but both improve the evidence without anyone annotating it.
| Comments: | 16 pages, 5 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.06962 [cs.CL] |
| (or arXiv:2610.06962v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.06962 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Prasang Gupta [view email]
[v1]
Sat, 3 Oct 2026 17:01:42 UTC (204 KB)
来源：arXiv：cs.AI · arxiv.org