arXiv：cs.AI· Boyuan Yang, Wei Gao·· 16 小时前
SchemaWeaver：通过 Schema 引导的并行解码加速端侧移动智能体 AI
SchemaWeaver: Accelerating On-Device Mobile Agentic AI via Schema-Guided Parallel LLM Decoding
AI 导读
SchemaWeaver 是一种端侧移动智能体 AI 技术，通过在 LLM 工具调用的解码阶段执行 Schema 引导的并行解码，每次解码生成多个 token 以降低延迟。它无需训练或修改 LLM 即可应用于移动端推理引擎，在主流移动 SoC 和 LLM 系列上相比竞争基线最高降低 3.9 倍工具调用延迟。
正文
Abstract:Large language models (LLMs) on mobile devices can act as agents that call external software tools to complete complex tasks, but the major challenge of supporting on-device mobile agentic AI with LLMs is the long latency of LLM tool calling on mobile devices. In particular, the decode phase is the main contributor of such long latency, but is hard to speed up on mobile processors as it is bound by memory bandwidth rather than compute power. To address this limitation, in this paper we present SchemaWeaver, a new on-device mobile agentic AI technique that enforces schema-guided parallel decoding in the decode phase of LLM tool calling, to reduce its latency by producing multiple tokens in each decode pass. SchemaWeaver can be generically applied to the LLM inference engine on mobile devices without involving any training efforts or changes on LLMs, and our experiment results on major mobile SoCs and LLM families show that SchemaWeaver can reduce the LLM tool calling latency by up to 3.9 times, compared to the competitive baselines.
| Comments: | 16 pages, 14 figures |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.17436 [cs.AI] |
| (or arXiv:2610.17436v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.17436 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Boyuan Yang [view email]
[v1]
Mon, 12 Oct 2026 21:53:20 UTC (624 KB)
来源：arXiv：cs.AI · arxiv.org