arXiv：cs.AI· Changguo Jia, Zhiyou Xiao, Xinbei Ma, Geng Chen, Weiming Zhang, Yiming Xu, Tianqi Zhao, Minghui Zhou·· 16 小时前
PaceReasoner：从长推理中实现无标签简短作答学习
PaceReasoner: Label-Free Learning to Answer Briefly from Long Reasoning
AI 导读
PaceReasoner 是一个无标签的测试时强化学习框架，利用模型的长推理响应提升其短模式准确率，两者共享同一可训练策略。在三种模型骨干和四个基准上，平均 SHORT 准确率较基线提升 7.4–13.5 个百分点，平均每条响应少于 25 个 reasoning token。
正文
Abstract:Extended chain-of-thought reasoning improves problem-solving performance but increases inference cost and latency, motivating more accurate answers with shorter reasoning. We introduce PaceReasoner, a label-free test-time reinforcement learning framework that uses a model's LONG responses to improve its SHORT-mode accuracy. Both modes share a trainable policy, creating a need for reliable supervision, sufficient SHORT training samples, and continued LONG responses to guide learning. PaceReasoner combines all-response and LONG-only answer consensus through progressive, confidence-weighted supervision. A behavior reward gradually shifts from encouraging SHORT exploration to rewarding agreement with LONG answers, while a ratio controller regulates the balance between modes. Across three model backbones and four benchmarks, PaceReasoner improves average SHORT accuracy over the baselines by 7.4--13.5 percentage points, with fewer than 25 reasoning tokens per response on average. Ablations support the key components, and cross-dataset results demonstrate transfer beyond the adaptation set.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.17900 [cs.AI] |
| (or arXiv:2610.17900v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17900 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Changguo Jia [view email]
[v1]
Tue, 13 Oct 2026 07:55:23 UTC (555 KB)
来源：arXiv：cs.AI · arxiv.org