ChatGPT 识别网络评论定向攻击与不当言论的效果评估与改进

View PDF HTML (experimental)

Abstract:This study evaluates the effectiveness of ChatGPT, an advanced AI model for natural language processing, in identifying targeting and inappropriate language in online comments. With the increasing challenge of moderating vast volumes of user-generated content on social network sites, the role of AI in content moderation has gained prominence. We compared ChatGPT's performance against crowd-sourced annotations and expert evaluations to assess its accuracy, scope of detection, and consistency. Our findings show that ChatGPT performs well in detecting inappropriate content, and that almost the entire gain from iterative prompt refinement came from a single change --- simplifying the label scheme and moving to GPT-4 --- after which further adjustments to prompt structure and sampling temperature had little effect. Its performance on targeting language was weaker and less stable, with higher false positive rates than expert judgments before refinement and higher false negative rates after a change of model. This study contributes to the field by demonstrating the potential of AI models like ChatGPT to enhance automated content moderation systems while also identifying areas for further improvement. The results underscore the importance of careful prompt design and model selection, and of contextual understanding, in supporting automated moderation and mitigating harmful online behavior.

Subjects:	Computation and Language (cs.CL)
Cite as:	arXiv:2505.21710 [cs.CL]
	(or arXiv:2505.21710v2 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2505.21710 arXiv-issued DOI via DataCite

Submission history

From: Baran Barbarestani [view email]
[v1] Tue, 27 May 2025 19:53:12 UTC (782 KB)
[v2] Tue, 13 Oct 2026 23:22:23 UTC (938 KB)