HICO-Bench：社交智能体的有效交互该如何衡量？用可验证结果训练 PaCPO

View PDF HTML (experimental)

Abstract:Effective social interaction requires more than fluent responses: agents must reach plans that fit the participants' needs and circumstances. Existing evaluations of social agents mainly rely on LLM judges that rate the conversation, without checking the resulting plan. We introduce HICO-Bench, a benchmark of 180 everyday coordination scenarios, such as resource allocation, scheduling, and authorization, in which two agents with private information converse and then independently record the plan they agreed on. A program checks Feasibility (the two records match and satisfy both participants' private conditions), and an LLM judge checks Integrity (the conversation supports the recorded plan); passing both is a Verified Success. Each scenario has two versions that share the same public context but differ in one private condition, so the correct plan differs between them. Across thirteen LLMs, 36% of conversations rated highly for goal completion fail Verified Success. To learn from outcomes that arrive only at the end of a conversation, we propose Paired-Credit Policy Optimization (PaCPO). After each turn, PaCPO asks the agent for a tentative plan, checks it in both versions of the scenario, and rewards the turns after which the tentative plans fit their respective versions. PaCPO raises Qwen3-8B's Verified Success from 20.4% to 28.3%.

Subjects:	Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Social and Information Networks (cs.SI)
Cite as:	arXiv:2610.20195 [cs.CL]
	(or arXiv:2610.20195v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.20195 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Caixin Kang [view email]
[v1] Wed, 14 Oct 2026 13:44:30 UTC (1,174 KB)