arXiv：cs.AI· Yuyu Liu, Haotian Xu, Yanan He, Sarang Rajendra Patil, Mengjia Xu, Tengfei Ma·· 4 小时前AI 评分41
HyperGuide：用双曲空间引导实现大语言模型高效多步推理
HyperGuide: Hyperbolic Guidance for Efficient Multi-Step Reasoning in Large Language Models
AI 导读
HyperGuide 是一个将推理视为双曲空间中状态树移动的框架，先在 Poincaré 球上训练状态编码器，再训练引导头预测最小代价转移方向，并以虚拟 token 形式插入解码器，使推理沿单一自回归轨迹进行，无需扩展或重排候选。在竞赛数学与代码生成任务上，它的准确率匹配或超过基于搜索和验证器的基线，同时生成的 token 数大幅减少，与 few-shot prompting 相当。
正文
Abstract:Searching over alternative continuations lets large language models (LLMs) compare the downstream consequences of a reasoning step before committing to it, but expanding and evaluating many branches makes inference expensive. Training value models to score intermediate steps does not avoid this cost, as they still rank candidates at inference time. We introduce HyperGuide, a framework that treats reasoning as movement through a tree of states embedded in hyperbolic space. HyperGuide first trains a state encoder on the Poincar'e ball, then trains a guidance head to predict the direction of a minimum-cost transition from the current state. The predicted direction is inserted into the decoder as a virtual token after each reasoning step, so inference follows a single autoregressive trajectory without expanding or reranking candidates. Because the supervision is ordinal rather than binary, it favors continuations that are both reliable and short, which steers generation toward successful, concise solutions. Experiments on competition mathematics and code generation show that HyperGuide matches or exceeds the accuracy of search- and verifier-based baselines while generating substantially fewer tokens than these baselines and about as many as few-shot prompting.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.24140 [cs.AI] |
| (or arXiv:2605.24140v5 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.24140 arXiv-issued DOI via DataCite |
Submission history
From: Yuyu Liu [view email]
[v1]
Fri, 22 May 2026 19:01:25 UTC (2,847 KB)
[v2]
Wed, 27 May 2026 23:12:50 UTC (2,848 KB)
[v3]
Sun, 2 Aug 2026 16:07:48 UTC (2,468 KB)
[v4]
Thu, 1 Oct 2026 04:34:36 UTC (303 KB)
[v5]
Fri, 2 Oct 2026 01:48:49 UTC (303 KB)
来源：arXiv：cs.AI · arxiv.org