arXiv：cs.LG（机器学习，全量分类）· Don Li·· 14 小时前AI 评分31
Q-MINO：面向量化感知训练的最小范数方法
Q-MINO: A Minimal-Norm Method for Quantization-Aware Training
AI 导读
研究者提出量化感知最小范数优化器 Q-MINO，用时间束方法结合梯度共识、状态漂移正则与对齐约束，从近期优化状态构造稳定的最小范数更新方向，以缓解 STE 代理梯度失配导致的噪声更新与参数振荡。该方法通过热启动 Frank-Wolfe 求解约束子问题，并借助随机 Lyapunov KL 框架证明其达到渐近邻域收敛。
正文
Abstract:The Straight-Through Estimator (STE) is a widely used heuristic for Quantization-Aware Training (QAT), but its surrogate gradients can exhibit substantial mismatch with the underlying quantized objective, leading to noisy updates and parameter oscillations, particularly in ultra-low-bit regimes. We propose the Quantization-Aware Minimal-Norm Optimizer (Q-MINO), a temporal bundle method that combines gradient consensus, state-drift regularization, and an alignment constraint to construct stabilized, minimum-norm update directions from recent optimization states. Q-MINO solves the resulting constrained subproblem using a warm-started Frank--Wolfe procedure with a feasible fallback initialization. Theoretically, via a stochastic Lyapunov Kurdyka--Łojasiewicz (KL) framework, we show that Q-MINO achieves asymptotic neighborhood convergence. Moreover, we detail numerical experiments with Q-MINO at various quantizations.
| Subjects: | Optimization and Control (math.OC); Hardware Architecture (cs.AR); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.00738 [math.OC] |
| (or arXiv:2610.00738v1 [math.OC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00738 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Don Li [view email]
[v1]
Wed, 30 Sep 2026 21:31:05 UTC (4,095 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org