ReplaySSM：重新思考混合模型解码的状态存储

ReplaySSM: Rethinking State Storage for Faster Hybrid-Model Decoding

AI 导读

NVIDIA 提出 ReplaySSM，重新思考混合模型解码中的状态存储方式。NVIDIA Nemotron 3、Qwen3.5 至 Qwen3.8、Kimi-K3 和 GLM-5.3-Flash 等混合模型将 SSM 层与少量注意力层交错，SSM 层以固定大小状态替代随序列增长的 KV cache。

正文

A cube assembled from smaller dark blocks, with glowing cyan, pink, yellow, and green sections against a black background. Hybrid models interleave state-space model (SSM) layers with a few attention layers. NVIDIA Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3, and GLM-5.3-Flash are...

Hybrid models interleave state-space model (SSM) layers with a few attention layers. NVIDIA Nemotron 3, Qwen3.5 through Qwen3.8, Kimi-K3, and GLM-5.3-Flash are all built this way, with Mamba-2, Gated DeltaNet (GDN), or Kimi Delta Attention (KDA) as the SSM layer. SSM layers are designed to make decoding fast: they use a fixed-size state in place of a key-value (KV) cache that grows with…

Source

来源：NVIDIA Technical Blog · developer.nvidia.com