arXiv：cs.CL· Ke Wan, Chen Chen·· 13 小时前
RAMP：面向循环深度语言模型高效 Prefill 的循环自适应混合精度
RAMP: Recurrent Adaptive Mixed Precision for Efficient Prefill in Recurrent-Depth Language Models
AI 导读
RAMP 是一种循环自适应混合精度方法，针对循环深度语言模型在长上下文 prefill 中因反复处理输入序列带来的额外开销，在循环深度早期使用低精度权重与激活计算、末端恢复高精度。在最长 4K 上下文下，RAMP 保持接近全精度的 prefill 质量，并在 NVIDIA H100 上实现最高 1.16 倍端到端首次 prefill 加速，对应 13.9% 延迟降低。
正文
Abstract:Recurrent-depth language models improve inference-time reasoning by repeatedly applying a shared network to refine internal representations, but the additional recurrent computation also increases inference cost. This overhead is particularly pronounced during long-context prefill, where the full input sequence is repeatedly processed across recurrent depth. Weight-and-activation quantization offers a natural way to accelerate this computation through lower-precision arithmetic, yet repeatedly applying quantized computation introduces numerical perturbations that accumulate along the recurrent trajectory. We systematically analyze how such errors propagate across recurrent depth and show that quantization errors introduced earlier are attenuated by more subsequent recurrent steps and therefore have a smaller effect on the final representation. Matched-budget experiments strongly support this prediction, showing that placing quantized computation earlier in recurrent depth consistently yields the best fidelity. Based on this insight, we introduce RAMP (Recurrent Adaptive Mixed Precision), which performs early recurrent refinement using low-precision weight-and-activation computation and restores high precision for terminal refinement. Across context lengths up to 4K, RAMP preserves near-full-precision prefill quality while achieving up to 1.16x end-to-end first-prefill speedup on an NVIDIA H100, corresponding to a 13.9% latency reduction.
| Comments: | 16 pages, 3 figures, 6 tables. Preprint |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.13041 [cs.CL] |
| (or arXiv:2610.13041v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13041 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ke Wan [view email]
[v1]
Fri, 9 Oct 2026 05:57:33 UTC (95 KB)
来源：arXiv：cs.CL · arxiv.org