arXiv：cs.CL· Rishitosh Kumar Singh, Amir Saeidi, Shamanthak Hegde, Shrinidhi Kumbhar, Chitta Baral·· 4 小时前
τ-Emotion-Bench：用真实用户情绪暴露 LLM 智能体的交互脆弱性
$\tau$-Emotion-Bench: Exposing the Interaction Fragility of LLM Agents through Real-World User Emotions
AI 导读
研究团队提出 τ-Emotion-Bench，一个覆盖十个领域、在情绪条件化用户交互下评测工具调用智能体的基准，并配套可扩展的数据生成管线 Tracer。实验显示前沿智能体的平均任务成功率与稳定完成率之间存在显著差距，Tracer 生成的监督数据可持续提升不同规模 Qwen3 模型，并泛化到未见领域及迁移至 τ-Bench 和 τ-Trait。
正文
Abstract:In long-horizon tool-calling tasks, LLM agents can fail not only because of tool-use errors, but also because of ineffective interaction with users. Yet existing benchmarks largely assume cooperative and behaviorally stable users, leaving the effect of emotional variation on agent reliability underexplored. We introduce $\tau$-Emotion-Bench, a ten-domain benchmark for evaluating tool-calling agents under controlled emotion-conditioned user interactions while preserving the underlying task objective. To scale both benchmark construction and agent training, we introduce Tracer, a scalable agentic data-generation pipeline that constructs grounded, policy-valid tasks and collects trajectories through an interactive agent--user--environment harness. By allowing dissatisfied users to terminate inefficient interactions, the harness provides an interaction-aware signal that filters trajectories with excessive user dependence and favors effective tool use. Experiments with two user simulators show that frontier agents remain unreliable, with substantial gaps between average task success and consistent task completion. Tracer-generated supervision consistently improves Qwen3 models across scales, generalizes to unseen domains, and transfers to $\tau$-Bench and $\tau$-Trait. Behavioral analyses further show that higher task success is associated with a better balance between tool execution and necessary user follow-up. Overall, $\tau$-Emotion-Bench and Tracer provide a scalable framework for evaluating agent reliability under diverse user behaviors and improving tool-calling agents through interaction-aware supervision.
| Comments: | Accepted to the UserSim Workshop at NeurIPS 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19560 [cs.AI] |
| (or arXiv:2610.19560v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19560 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Seyyedamirhossein Saeidi [view email]
[v1]
Wed, 14 Oct 2026 04:19:15 UTC (4,244 KB)
来源：arXiv：cs.CL · arxiv.org