arXiv：cs.LG（机器学习，全量分类）· Tilen Cadez, Sanghoon Lee, Kyoung-Min Kim·· 1 天前AI 评分32
Kolmogorov-Arnold 网络（KAN）的神经缩放定律与可学习激活函数演化
Neural scaling laws and evolution of learnable activation functions of Kolmogorov-Arnold networks
AI 导读
研究考察了 BSRBF-KAN、Gottlieb-KAN 和 Faster-KAN 三种 KAN 变体的神经缩放定律：在 MNIST、Fashion-MNIST 图像分类任务上。
正文
Abstract:Kolmogorov-Arnold Networks (KANs) represent a compelling alternative to traditional Multi-Layer Perceptron (MLP)-based neural networks. By employing activation functions as learnable elements, KANs offer superior interpretability, making them suited for scientific domains. In this work, we investigate the neural scaling laws of KANs and the structural evolution of their learnable activation functions under dataset expansion. Specifically, we evaluate the scaling behavior of three KAN variants---BSRBF-KAN, Gottlieb-KAN, and Faster-KAN---across standard image classification benchmarks (MNIST and Fashion-MNIST) and a specialized scientific regression task (magnetic parameter estimation from domain images of moiré magnetic textures). Our results demonstrate that the test loss ${\cal L}$ exhibits a broken neural scaling law (BNSL) behavior as a function of the dataset size $N_D$. After passing through a random-guess regime, the loss follows architecture- and task-dependent scaling behavior. The loss crosses from a faster- to a slower-scaling branch, ${\cal L}\propto N_D^{-\alpha}$ and ${\cal L}\propto N_D^{-\beta}$ with $\alpha>\beta$ for image classification tasks. The exponents $\alpha$ and $\beta$ depend strongly on both the specific network architecture and the dataset-size regime, ranging from 0.4 to 1.5 and from 0.06 to 0.6, respectively. For the magnetic parameter-regression task, the loss follows a single scaling law with its exponent ranging from 1.28 to 2.59. Additionally, we provide a structural analysis of how activation functions refine their complexity as data volume increases, finding that dataset expansion drives a transition from simple linear-like approximations toward stable, interpretable symbolic forms. These findings provide a quantitative roadmap for the efficient application of KANs while managing the trade-off between model expressivity and computational overhead.
| Comments: | 13 pages, 10 figures. Supplementary Notes will be provided in the published version |
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.00985 [cs.LG] |
| (or arXiv:2610.00985v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00985 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kyoung-Min Kim Prof [view email]
[v1]
Thu, 1 Oct 2026 03:22:11 UTC (4,239 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org