arXiv：cs.LG· Fan Gao, Wei Su, Juntong Fan, Renfeng Peng, Hongyu Liu, Jinqiao Duan, Feng-Lei Fan·· 4 小时前
动态系统即代码：用动力学系统做模型压缩
Dynamics as Code: On Model Compression via Dynamic System
AI 导读
一项研究提出将模型压缩重新定义为紧凑权重表示：用动态系统生成轨迹的索引来编码高维参数，解压时再还原向量，机制区别于剪枝、量化、知识蒸馏和低秩分解。作者在 Diophantine 条件下证明无理缠绕的有限轨迹可构成 d 维权重空间的 ε-net，并统一空间填充曲线、混沌系统、同余与伪随机生成器、低差异序列四类动态系统，引入 KD-tree 与坐标模板加速及离群点识别控制误差。
正文
Abstract:The escalating size of pretrained neural networks has rendered model compression a prerequisite for deployment under stringent memory and compute constraints. With the irrational winding as an example, earlier work introduced a dynamic system (DS) paradigm that reconceptualizes compression as compact weight representation: high-dimensional parameters are encoded by the index of a trajectory produced by a dynamic system, from which the vector is recovered during decompression. This mechanism is fundamentally distinct from pruning, quantization, knowledge distillation, and low-rank decomposition. Along this direction, we prove that under a Diophantine condition, a finite trajectory of \(M = O(\epsilon^{-(d+\nu)})\) states in the irrational winding constitutes an \(\epsilon\)-net over the \(d\)-dimensional weight space, thereby linking state resolution, decompression error, and compression ratio in a predictable manner. Furthermore, we propose a generalized DS-based model compression framework by unifying four DS families---space-filling curves (Hilbert, Peano, Morton/Z-order, Snake), chaotic systems (Lorenz), congruential and pseudo-random generators (LCG, PCG), and low-discrepancy sequences (Halton). Also, we introduce the KD-tree and coordinate-template acceleration to scale to large models as well as outlier identification to control the error. Experiments on ResNet-18 and Qwen2.5-1.5B/Qwen1.5-7B validate that DS-based compression achieves competitive compression ratios without post-hoc retraining, with controllable decompression error and flexible state-space design, establishing it as a principled and practical compression approach.
| Comments: | 17 pages including supplementary material |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.11115 [cs.LG] |
| (or arXiv:2610.11115v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.11115 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Fan Gao [view email]
[v1]
Thu, 8 Oct 2026 02:39:24 UTC (19,359 KB)
来源：arXiv：cs.LG · arxiv.org