arXiv：cs.LG· Hanzhi Zhang, Qiao Zhang, Qinglei Cao, Heng Fan, Yan Huang, Kewei Sha, Yunhe Feng·· 4 小时前AI 评分43
AlignQuant：面向高效 LLM 生成的 Tile 对齐混合精度量化
AlignQuant: Tile-Aligned Mixed-Precision Quantization for Efficient LLM Generation
AI 导读
AlignQuant 是一种训练后量化方法，以 GPU 兼容的二维权重 tile 作为精度分配、紧凑存储与执行的统一单元。该方法在 3B 至 14B 四个 LLM 上实现最高 2.50× 的生成加速（对比 BF16）并保持模型质量，评测覆盖三种 GPU 与最高 64K token 上下文。代码已开源。
正文
Abstract:Fine-grained mixed-precision quantization promises efficient large language model inference, but local precision choices can conflict with regular GPU storage and computation units. This precision-boundary mismatch limits the translation of compression into practical acceleration. We introduce AlignQuant, a post-training quantization method that uses GPU-compatible two-dimensional weight tiles as the common unit of precision allocation, compact storage, and execution. This shared partition lets precision follow sensitivity within output channels. Joint prefill/decode calibration scores precision reductions using projection-output perturbations weighted by language-model loss gradients under quantized activations. Phase-normalized scores prioritize higher precision for tiles important to either phase under a model-wide weight-storage budget. Each tile stores one selected representation, while phase-specialized kernels reuse the packed model and expand lower-bit weights for INT8 computation with 8-bit activations. Across four LLMs spanning 3B to 14B parameters, AlignQuant achieves up to $2.50\times$ generation speedup over BF16 while preserving model quality. Evaluations further cover three GPUs and contexts up to 64K tokens. These results show that local precision flexibility and regular GPU execution can coexist through a shared tile unit. The implementation is available at this https URL.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.07457 [cs.LG] |
| (or arXiv:2610.07457v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.07457 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hanzhi Zhang [view email]
[v1]
Mon, 5 Oct 2026 22:06:04 UTC (581 KB)
来源：arXiv：cs.LG · arxiv.org