arXiv：cs.AI· Tianyou Lai, Wentao Yue, Zhiying Deng, Qilei Li·· 16 小时前
Choice2Seq：衡量模型跨输出接口的正确性保持能力
Correct Choices, Failed Sequences: Choice2Seq Benchmarks Correctness Retention Across Output Interfaces
AI 导读
Choice2Seq 是一个诊断性基准，在相同问题和候选集上独立测量模型"完整答案选择"与"序列化构建"两类任务间的正确性保持度，指标为 SCR。Jev 在算术与词汇检索上的选择准确率分别为 97.1% 和 99.5%，但序列构建下仅保留 33.0% 和 2.1% 的正确选择；StartLux 与 Qwen 也出现显著差距，即使 Qwen 直接输出字符时依然存在。
正文
Abstract:A model may select the correct answer yet fail to construct it through a sequence of local decisions. We introduce Choice2Seq, a diagnostic benchmark that measures correctness retention across complete-answer selection and sequential construction in independent runs on the same problems and candidate sets. Sequence Composition Rate (SCR) measures the fraction of correctly selected items that are also answered correctly through local construction. Jev achieves 97.1% and 99.5% selection accuracy on arithmetic and lexical retrieval, yet retains only 33.0% and 2.1% of those correct choices under serial construction, respectively. Substantial gaps also emerge for StartLux and Qwen, including when Qwen emits characters directly. Restricting local options to candidate-supported characters improves construction, while explicit position indexing improves copying fidelity, even for incorrect selections. These interventions distinguish task correctness, plan fidelity, and output compliance. Matched dialogue-state evaluations reveal that local field construction largely preserves Jev's correct states but reduces Qwen's mean-turn accuracy, highlighting task- and interface-dependent differences. Sequence-level confidence and request-cost analyses further characterize completed outputs.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.16191 [cs.AI] |
| (or arXiv:2610.16191v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16191 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tianyou Lai [view email]
[v1]
Mon, 12 Oct 2026 05:11:37 UTC (947 KB)
来源：arXiv：cs.AI · arxiv.org