arXiv：cs.AI· Xiangyu Li, Huaizhi Tang, Xin Ding, Weijun Wang, Ting Cao, Yunxin Liu·· 5 小时前
OxyGen：面向多任务并行下 VLA 推理的统一 KV Cache 管理
OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelism
AI 导读
OxyGen 提出统一 KV Cache 管理，将 KV Cache 视为跨任务、跨控制周期的一级共享资源，通过跨任务 KV 共享消除共享观测的冗余 prefill，并用跨帧连续批处理在每帧解码预算内推进未完成请求。
正文
Abstract:Embodied AI agents increasingly require parallel execution of multiple tasks, such as manipulation, conversation, and memory construction, from shared observations under distinct time constraints. Recent Mixture-of-Transformers (MoT) Vision-Language-Action Models (VLAs) support such heterogeneous outputs, yet existing inference systems incur redundant computation and resource contention when serving concurrent tasks on-device. We identify isolated KV cache management as the root cause. To address this, we propose unified KV cache management, an inference design that treats the KV cache as a first-class shared resource across tasks and control cycles. This abstraction enables two key optimizations: cross-task KV sharing eliminates redundant prefill of shared observations, while cross-frame continuous batching batches unfinished language requests from different frames and advances them within a per-frame decoding budget. We implement this design in OxyGen and evaluate $\pi_{0.5}$ on NVIDIA GeForce RTX 4090 and Jetson AGX Thor. OxyGen achieves up to 3.7$\times$ speedup over isolated execution, delivering over 200 tokens/s language throughput and 70 Hz action frequency simultaneously on RTX 4090. We also validate the design on Xiaomi-Robotics-0 and StarVLA across different backbones and expert configurations. We further demonstrate the inference gains on a humanoid robot with on-board Jetson AGX Thor.
| Comments: | NeurIPS 2026 |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2603.14371 [cs.RO] |
| (or arXiv:2603.14371v4 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2603.14371 arXiv-issued DOI via DataCite |
Submission history
From: Xiangyu Li [view email]
[v1]
Sun, 15 Mar 2026 13:23:56 UTC (645 KB)
[v2]
Mon, 18 May 2026 10:25:25 UTC (4,764 KB)
[v3]
Tue, 13 Oct 2026 11:14:49 UTC (5,893 KB)
[v4]
Wed, 14 Oct 2026 06:30:19 UTC (1,906 KB)
来源：arXiv：cs.AI · arxiv.org