arXiv：cs.AI· Michael Kleinman, Matthew Trager, Alessandro Achille, Wei Xia, Stefano Soatto·· 21 小时前
e1：让模型学会自适应控制推理努力
e1: Learning Adaptive Control of Reasoning Effort
AI 导读
研究者提出 Adaptive Effort Control，一种自适应强化学习方法，让模型按用户指定的 token 比例（相对当前平均思维链长度）分配推理量，无需按数据集和阶段调超参，用户在推理时用连续 effort 参数即可调节成本与准确率权衡。在 1.5B 至 32B 参数规模上，该方法在数学任务中保持同等准确率的同时，将思维链长度较 RL 训练基座模型缩短 1.9-3.7 倍。
正文
Abstract:Increasing the thinking budget of AI models can significantly improve accuracy, but not all questions warrant the same amount of reasoning. Users may prefer to allocate different amounts of reasoning effort depending on how they value output quality versus latency and cost. To leverage this tradeoff effectively, users need fine-grained control over the amount of thinking used for a particular query, but few approaches enable such control. Existing methods require users to specify the absolute number of desired tokens, but this requires knowing the difficulty of the problem beforehand to appropriately set the token budget for a query. To address these issues, we propose \textit{Adaptive Effort Control}, a self-adaptive reinforcement learning method that trains models to use a user-specified fraction of tokens relative to the current average chain-of-thought length for each query. This approach eliminates dataset- and phase-specific hyperparameter tuning. Users can dynamically adjust the cost-accuracy trade-off through a continuous effort parameter specified at inference time. We observe that the model automatically learns to allocate resources proportionally to the task difficulty. Across model scales ranging from 1.5B to 32B parameters, our approach reduces chain-of-thought length on math tasks by 1.9-3.7x at matched accuracy relative to the base model used for RL training.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2510.27042 [cs.AI] |
| (or arXiv:2510.27042v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2510.27042 arXiv-issued DOI via DataCite |
Submission history
From: Michael Kleinman [view email]
[v1]
Thu, 30 Oct 2025 23:12:21 UTC (369 KB)
[v2]
Wed, 12 Nov 2025 04:15:18 UTC (369 KB)
[v3]
Sat, 10 Oct 2026 00:49:04 UTC (386 KB)
来源：arXiv：cs.AI · arxiv.org