arXiv：cs.AI· Zeyu Yang, Wes Armour·· 16 小时前
开源权重 LLM 推理能效实测：跨 benchmark、推理模式与硬件平台
Measuring the Inference Energy Efficiency of Open-Weight LLMs across Benchmarks, Reasoning Modes, and Hardware
AI 导读
研究直接测量了 86 个开源权重模型 checkpoint 在七个 benchmark 和六种硬件平台（从消费级 GPU 到八卡服务器）上的推理能效。多数配置远离分数-能耗 Pareto 前沿，能耗中位数比同等分数配置高 2.6-4.8 倍，最高达 27 倍；开启思考模式平均增加约 3.7 倍能耗，FP8 则降低约四分之一能耗且分数变化不大。
正文
Abstract:Benchmark scores tell us what an LLM can do, but not how much energy is required to deliver that capability. We directly measure this trade-off for 86 open-weight model checkpoints, evaluated on seven benchmarks and six hardware platforms ranging from a consumer GPU to an eight-GPU server. We find large differences in energy efficiency even among models with similar capability. Most configurations lie away from the score--energy Pareto frontier: they use a median of 2.6--4.8$\times$ more energy than another configuration that scores at least as well, with differences reaching 27$\times$. Reasoning can improve capability, but enabling thinking increases energy by about 3.7$\times$ on average, and the benefit depends strongly on the model family and output budget. Generated-token count is also an imperfect proxy for energy, misordering 18--24\% of model pairs. In contrast, FP8 reduces energy by roughly a quarter with little change in score, while changing hardware largely rescales energy without changing the relative ranking of models. These results show that capability alone gives an incomplete picture of efficiency. We argue that benchmark scores should be reported together with measured energy consumption under a specified serving regime and output budget.
| Subjects: | Artificial Intelligence (cs.AI); Performance (cs.PF) |
| Cite as: | arXiv:2610.14389 [cs.AI] |
| (or arXiv:2610.14389v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14389 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zeyu Yang [view email]
[v1]
Fri, 9 Oct 2026 23:15:10 UTC (626 KB)
来源：arXiv：cs.AI · arxiv.org