arXiv：cs.AI· Keisuke Kamahori, Shihang Li, Simon Peter, Baris Kasikci·· 16 小时前
VibeServe：AI 智能体能否构建定制化 LLM 服务系统？
VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?
AI 导读
VibeServe 是首个端到端生成完整 LLM 服务栈的智能体循环，通过外层循环规划搜索系统设计、内层循环实现候选方案并验证正确性与性能。在标准场景下，VibeServe 在 Llama-3.1-8B 上持平 vLLM 和 SGLang；在四块 AMD MI300A 上服务 397B MoE 模型时，有效吞吐量（goodput）达调优 SGLang 的 2.3 倍。
正文
Abstract:For years, we have built LLM serving systems like any other critical infrastructure, with a single general-purpose stack hand-tuned over many engineer-years to support every model and workload. In this paper, we take the opposite bet and let coding agents automatically synthesize bespoke serving systems for different usage scenarios. We propose VibeServe, the first agentic loop that generates entire LLM serving stacks end-to-end. VibeServe uses an outer loop to plan and track the search over system designs, and an inner loop to implement candidates, check correctness, and measure performance on the target benchmark. In standard serving settings, VibeServe matches vLLM and SGLang on Llama-3.1-8B, and delivers 2.3x the goodput of a tuned SGLang for a 397B MoE model on four AMD MI300As. In five non-standard scenarios, VibeServe outperforms existing systems by exploiting opportunities that generic systems miss, spanning non-standard model architectures, workload knowledge, and hardware-specific optimizations. Together, these results argue that infrastructure software should favor generation-time specialization over runtime generality. Code is available at this https URL.
| Subjects: | Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC) |
| Cite as: | arXiv:2605.06068 [cs.AI] |
| (or arXiv:2605.06068v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.06068 arXiv-issued DOI via DataCite |
Submission history
From: Keisuke Kamahori [view email]
[v1]
Thu, 7 May 2026 11:54:53 UTC (738 KB)
[v2]
Mon, 12 Oct 2026 08:54:58 UTC (970 KB)
来源：arXiv：cs.AI · arxiv.org