arXiv：cs.AI· Gordon V. Cormack, Maura R. Grossman, Jeremy Pickens·· 20 小时前
研究：Relativity 生成式 AI 审查工具 aiR 与人工审查效果相当，CAL 模型仅用 2000 条样本即达 SOTA
Comparing GenAI to Machine Learning and Manual Review for eDiscovery
AI 导读
针对 Redgrave LLP 案例研究中 aiR 达 88% recall、RAL 达 64% recall 的结论，该分析指出两者方法目标不同、评估者间存在分歧，指标不可直接比较，计入这些差异后 aiR 与人工审查效果相当。采用 Continuous Active Learning（CAL）的成熟机器学习模型仅用 2000 条首轮审查员相关性判定样本训练，即展现出 SOTA 效果与更高效率。
正文
Abstract:Keeling et al. [Keeling2026Generative] report on a case study recently conducted by Redgrave LLP in which they assert that Relativity's Generative AI Review tool (aiR) achieved 88% recall while Relativity Assisted Learning (RAL), its machine-learning tool, achieved 64% recall. These results are not comparable because the methods employed have different objectives, because of inter-assessor disagreement, and because they represent different recall-precision and cost-benefit trade-offs. Our analysis of the Redgrave data indicates that, when these differences are taken into account, aiR and manual review are comparable. Furthermore, established machine-learning models employing Continuous Active Learning (CAL) show state-of-the-art effectiveness, and superior efficiency, when trained only on a sample of 2,000 first-pass-reviewer responsiveness determinations.
| Comments: | 8 pages; 2 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2610.16925 [cs.LG] |
| (or arXiv:2610.16925v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16925 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Gordon Cormack [view email]
[v1]
Mon, 12 Oct 2026 15:08:40 UTC (63 KB)
来源：arXiv：cs.AI · arxiv.org