arXiv：cs.LG（机器学习，全量分类）· Fabio J. Fehr, Philip Torr·· 14 小时前AI 评分33
超网络即时生成权重：ARC-1D 上的概念验证
On-the-fly Weight Generation: A Hypernetwork Proof of Concept on ARC-1D
AI 导读
研究以 ARC-1D 为受控测试平台，验证超网络可从少量演示中即时生成专用模型的参数，将少样本任务上下文编译为紧凑的可执行模型权重。生成的参数形成结构化权重空间，所得专用模型展现出部分组合泛化能力，并能泛化到训练中未见过的变换；在两种设定下，移除显式任务标识符都能提升超出训练变换的泛化表现。
正文
Abstract:General-purpose models can adapt to many tasks from context, while specialised models can execute individual functions with less capacity. Yet obtaining such specialists requires task-specific training or adaptation. We ask whether they can instead be generated directly from a few demonstrations. Using ARC-1D as a controlled testbed, we show that individual transformations can be represented by tiny specialist models, and that a hypernetwork can generate their parameters from context. The generated parameters form a structured weight space, while the resulting specialists show partial compositional generalisation and generalisation to transformations not seen during training. In both settings, removing explicit task identifiers improves generalisation beyond the training transformations. Together, these results provide a proof of concept that few-shot task context can be compiled on-the-fly into compact executable model parameters, and that the resulting weight space can support reuse and generalisation beyond known functions.
| Comments: | Published (Spotlight) at NeurIPS 2026 Workshop on Neural Network Artifacts as a New Data Modality |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.00820 [cs.LG] |
| (or arXiv:2610.00820v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.00820 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Fabio James Fehr [view email]
[v1]
Wed, 30 Sep 2026 23:26:25 UTC (888 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org