arXiv：cs.CL· Ali Zamani, Soroush Esfahanian, Erfan Shahabi, Mohammad Taher Pilehvar, Azadeh Shakery, Yadollah Yaghoobzadeh·· 16 小时前
PerSo：在真实劝服场景中考察 LLM 社会智能的多个维度
PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios
AI 导读
PerSo 是一个基于 Reddit ChangeMyView 真实劝服对话构建的基准，用 LLM-in-the-loop 流程生成评测任务并经人工标注验证，同时考察 LLM 的认知智能与行为智能。评测显示模型表现总体随规模提升但各维度不一：最强 LLM 在行为任务上略超最佳人类表现，而认知任务上最佳人类参与者仍领先。
正文
Abstract:Understanding the social intelligence of large language models (LLMs) is increasingly important for applications involving human-AI interaction and multi-agent environments. Social intelligence comprises two aspects: cognitive intelligence, the ability to infer others' mental states, and behavioral intelligence, the ability to select appropriate social actions based on these inferences to achieve social goals. Persuasion provides a natural testbed in which both aspects are simultaneously at play, yet prior benchmarks typically evaluate them in isolation using synthetic scenarios. We introduce PerSo, a benchmark that evaluates both aspects on real-world persuasion dialogues from Reddit's ChangeMyView community, using an LLM-in-the-loop pipeline to construct the evaluation tasks, validated through human annotation. Our evaluation of diverse LLMs shows that performance generally scales with model size but varies across dimensions. The strongest LLM slightly surpasses the best observed human performance on the behavioral task, while the best human participant remains ahead on the cognitive task.
| Comments: | 25 pages, 17 figures, 10 tables. Ali Zamani and Soroush Esfahanian contributed equally |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Social and Information Networks (cs.SI) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.20548 [cs.CL] |
| (or arXiv:2610.20548v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20548 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Erfan Shahabi [view email]
[v1]
Wed, 14 Oct 2026 17:39:35 UTC (5,138 KB)
来源：arXiv：cs.CL · arxiv.org