NVIDIA 通过新的 vLLM 上游优化，在 2.8T 参数 Kimi K3 模型上实现 423 tok/s/user，同硬件提速 1.68x。该优化由 NVIDIA CentML 的 Benjamin 用 Cute DSL 编写解码专用 kernel，采用激进 Megakernel 融合，达到 12,000 行代码。NVIDIA 还在为 vLLM 推进 DeepSeekv4.1 等模型的支持。
BREAKING NEWS🚨: NVIDIA has just achieved 423 tok/s/user 🚀 on the 2.8T Parameter Kimi K3 model via a new upstream vLLM optimization. 1.68x Faster on the Same Hardware!
This was done by NVIDIA CentML's Benjamin, who wrote NVIDIA's new decode-specialized kernels in Cute DSL via aggressive Megakernel fusion. This is a 12,000 LoC Megakernel and is the future of advanced AI-assisted kernel generation. NVIDIA is already working on DeepSeekv4.1 and other models for vLLM too.
Someone should give this NVIDIA 10x engineer a raise for this great new major feature!
来源：SemiAnalysis · x.com