arXiv：cs.LG· Junlin Liao, Harshit Kondle, Di Liu, Zonghua Gu·· 3 小时前
边缘平台上的视觉语言模型推理能耗实测：Qwen2-VL-2B 在 Raspberry Pi 5 与 Jetson Orin Nano 上的表现
Seeing at the Edge: Energy Characterization of Vision-Language Model Inference on Edge Platforms
AI 导读
研究实测了 Qwen2-VL-2B-Instruct 及其纯文本版本在 Raspberry Pi 5（Hailo-10H NPU）和 Jetson Orin Nano Super 上的总系统能耗、延迟与热行为。
正文
Abstract:Vision-language models (VLMs) are moving onto edge devices, but little is known about how much energy they draw on single-board computers with NPU or GPU accelerators, or where that energy goes. We measure total-system energy, latency, and thermal behavior of Qwen2-VL-2B-Instruct and its text-only sibling on a Raspberry Pi 5 with a Hailo-10H NPU and on a Jetson Orin Nano Super, on both CPU and accelerator, and separate the fixed energy of each request from the energy that grows with output length. Because the accelerators use different quantization and serving software, we compare deployment stacks rather than silicon. Three findings stand out. First, NPU offloading makes the Pi a much more efficient VLM platform: it cuts energy per generated token by about two-thirds, leaves the host CPU largely free, and avoids the thermal throttling that the CPU backend reaches in sustained use. Second, an image adds both fixed and per-token energy, but at typical response lengths generating the answer costs far more than processing the image. Third, the Jetson GPU favors short responses through its lower fixed cost per request, whereas the Pi NPU favors longer ones through its lower cost per output token; their energy curves cross near 40 output tokens.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.20136 [cs.LG] |
| (or arXiv:2610.20136v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20136 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zonghua Gu [view email]
[v1]
Wed, 14 Oct 2026 12:55:12 UTC (364 KB)
来源：arXiv：cs.LG · arxiv.org