arXiv：cs.AI· Zhinan Hou, Xingchen Li, Keyou You·· 3 小时前
无需训练即可高效推理：通过隐空间约束优化缓解大模型过度思考
Efficient Reasoning via Constrained Optimization in Latent Space
AI 导读
研究者提出一种免训练框架，通过二次规划将偏离的隐藏状态投影回隐空间中高效推理步骤聚集的集中区域，从而在不牺牲性能的前提下减少token生成。在四个1.5B至14B模型、六个数学推理/编程/科学QA基准上，准确率最高提升12.1%，生成token减少11.8%至52.8%。该工作已被NeurIPS2026接收。
正文
Abstract:Large Reasoning Models (LRMs) have shown remarkable reasoning capabilities, yet they still suffer from overthinking, generating redundant reasoning steps which incur substantial token consumption. Existing methods, such as suppressing reflective keywords or forcing shorter reasoning lengths, attempt to mitigate this issue but inevitably truncate necessary steps and induce underthinking, thereby compromising performance. To address this dilemma, we investigate the latent representations and observe that efficient reasoning steps naturally cluster into a concentrated region in latent space, while those deviating from this region tend to produce verbose sequences. To leverage this, we keep reasoning focused within this region via a quadratic program which projects deviating hidden states back into the region. Then we propose a novel training-free framework to achieve efficient reasoning that reduces token generation costs without sacrificing performance. Extensive experiments conducted on four models ranging from 1.5B to 14B, and across six benchmarks in math reasoning, coding, and scientific QA, validate the effectiveness of our method, up to a 12.1\% improvement in accuracy while reducing generated tokens by 11.8\% to 52.8\%. Codes are available at \href{this https URL}{this https URL}.
| Comments: | Accepted by NeurIPS2026 |
| Subjects: | Artificial Intelligence (cs.AI); Optimization and Control (math.OC) |
| Cite as: | arXiv:2609.34181 [cs.AI] |
| (or arXiv:2609.34181v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.34181 arXiv-issued DOI via DataCite |
Submission history
From: Zhinan Hou [view email]
[v1]
Mon, 28 Sep 2026 02:59:01 UTC (10,569 KB)
[v2]
Thu, 8 Oct 2026 16:41:45 UTC (10,569 KB)
来源：arXiv：cs.AI · arxiv.org