SemiAnalysis 转发 SGLang 发布的 Low-Latency Mode，宣布 Megakernel 支持已合入，在 GB300 NVL72 的 AgentX Harness 上单用户推理速度接近 1000 tok/s，MI355X 交互性提升 1.5 倍。该功能已在 SGLang Docker 镜像中开箱即用。据 lmsysorg 引文，DeepSeek-V4.1-Flash with DSpark 在 NVIDIA GB300 上 P90 单用户 token 速度提升 1.38 倍（671 到 928 tok/s），AMD MI355X 提升 1.56 倍（332 到 516 tok/s），并新增支持外部库自定义内核自动回退的插件系统，GLM-5.3、K3、MiniMax M3 的低延迟内核在开发中。
ALERT🚨: SGLANG JUST MERGED MEGAKERNEL SUPPORT, GETTING CLOSE TO 1000 tok/s/user on the GB300 NVL72 AgentX Harness & increasing interactivity by 1.5x on MI355X too! Great work by the SGLang, RadixArk, NVIDIA, and AMD teams!
Best of all, this already works out of the box on SGLang Docker images! Excited to rapidly get this on the official AgentX!
https://x.com/lmsysorg/status/2112342810376712527
Introducing SGLang Low-Latency Mode, built with @NVIDIAAI and @AMD. On DeepSeek-V4.1-Flash with DSpark, it delivers up to 1.38× P90 per-user token speed on NVIDIA GB300 (671 to 928 tok/s) and 1.56× on AMD MI355X (332 to 516 tok/s) in @SemiAnalysis_'s AgentX trace replay. Highlights: - NVIDIA: overlapping kernels cut decoder layer time from 96.8 to 68.4 µs - AMD: 2 persistent kernels per layer replace dozens of small ones, with cross-GPU communication overlapped inside them. - New plugin system: custom kernels plug in from external libraries, with automatic fallback. - Low-latency kernels for GLM-5.3, K3, and MiniMax M3 are in progress and will plug in next.在 X 查看被引用的帖子
来源：SemiAnalysis · x.com