arXiv：cs.CL· Xing Li, Jinzhong Ning, Yijia Zhang, Liang Yang, Hongfei Lin·· 4 小时前
Jev 能否理解立场？Jev 与通用 LLM 的立场检测对比评测
Can Decision Models Understand Stance? Evaluating Jev Against General-Purpose LLMs
AI 导读
研究在 VAST（英文文本）和 ZS-CSD（中文对话）两个立场检测数据集上评测专用决策模型 Jev，并与四个通用 LLM 及两个微调模型对比。Jev 在 VAST 上表现与 GPT-5.6 相当，优于其他通用 LLM，但在 ZS-CSD 上落后于更强 LLM，尤其在区分"支持"与"反对"时。分析显示该局限可能与理解回复关系和立场方向有关，而非仅取决于对话长度。
正文
Abstract:Stance detection requires identifying an author's attitude toward a given target, sometimes based on conversational context. Jev, a specialized decision model designed for structured decision-making, offers an alternative to general-purpose large language models (LLMs). In this work, we evaluate Jev on two stance detection datasets, VAST (English texts) and ZS-CSD (Chinese conversations), comparing it with four general-purpose LLMs and two fine-tuned models. Results show that Jev achieves competitive performance on VAST, matching GPT-5.6 and outperforming the other general-purpose LLMs. However, it falls behind stronger LLMs on ZS-CSD, particularly in distinguishing favor from against. Further analysis suggests that this limitation may be related to understanding reply relationships and stance direction rather than conversation length alone. These findings highlight both the potential and limitations of Jev for stance detection.
| Comments: | 8 pages, 1 figure, 5 tables |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.11901 [cs.CL] |
| (or arXiv:2610.11901v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11901 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xing Li [view email]
[v1]
Thu, 8 Oct 2026 13:05:52 UTC (59 KB)
来源：arXiv：cs.CL · arxiv.org