arXiv：cs.CL· Demetris Paschalides, George Pallis, Marios D. Dikaiakos·· 4 小时前AI 评分34
HATEDECIDE：结构化决策模型用于仇恨言论审核的准确性与效率评估
To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation
AI 导读
研究团队提出 HATEDECIDE，在四个仇恨言论数据集上评估六种结构化决策模型配置，并与专用审核模型、零样本、商业 LLM 及监督基线对比。商业 LLM 仅在其中一个数据集上显著优于所有决策模型；提供数据集定义最多改变 28% 的预测但未持续提升分类效果，问题分解仅在 20% 的对比中显著改善性能。
正文
Abstract:The scale of online content makes hate-speech moderation challenging, while Large Language Models (LLMs) enable harmful material to be produced and adapted more easily. Moderation therefore requires efficient classifiers that can accommodate different definitions of hate speech. Recent structured decision models accept natural-language criteria and select among specified answers, raising the question of whether they can meet these requirements without task-specific training. We present HATEDECIDE, an evaluation of six decision-model configurations on four hate-speech datasets against specialized moderation, zero-shot, commercial, and supervised baselines. We examine whether supplying a dataset's definition, or decomposing it into multiple questions, improves classification, and we measure their latency and cost. We find that commercial LLMs significantly outperform all decision models on only one dataset. Supplying definitions changes up to 28\% of predictions without consistently improving classification, and decomposition significantly improves performance in only 20\% of the comparisons. On a diagnostic set of test cases, the best hosted decision model comes within 1.6 macro-F1 points of the best commercial LLM at approximately 97\% lower inference cost. These results identify opportunities for inexpensive moderation, while showing that explicit criteria and additional questions do not reliably improve classification.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.03324 [cs.CL] |
| (or arXiv:2610.03324v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03324 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Demetris Paschalides [view email]
[v1]
Fri, 2 Oct 2026 13:59:21 UTC (565 KB)
来源：arXiv：cs.CL · arxiv.org