arXiv：cs.LG（机器学习，全量分类）· Yuan Huang, Zihan Chen, Runbin Zhang, Hongwei Ding, Changzeng Fu, Shiqi Zhao·· 17 小时前AI 评分35
持续学习该在哪里特化：放置搜索与大脑读出殊途同归
Two Routes to the Middle: Placement Search and Brain Readouts Converge on Where Continual Learners Should Specialize
AI 导读
研究对比了持续学习中任务专属适配器的两种放置策略：穷举所有连续四块放置会得到倒 U 形曲线，最终准确率在中间深度达到峰值，波动最高达 3.5 个百分点；而基于权重谱或激活统计的低成本准则却偏好最深层。
正文
Abstract:Continual learners that keep a task-specific adapter in every block of a pre-trained vision transformer accumulate storage linearly with the number of tasks; keeping task-specific adapters in only a few blocks curbs this growth but raises the question of where to place them. We investigate this question from two perspectives. Algorithmically, training all contiguous four-block placements yields an inverted U: final accuracy peaks at intermediate depth and varies by up to 3.5 percentage points (pp), while inexpensive criteria based on weight spectra or activation statistics favor the deepest blocks. From neuroscience, the hierarchical organization and intermediate-stage plasticity of the visual cortex motivate us to ask whether a measurement taken outside the learner can guide layer specialization without placement search. LS-B observes the first tasks through a frozen fMRI encoding model of twelve human visual areas and commits task-specific capacity once to the blocks whose readouts vary most across tasks relative to their stable structure. Across three ViT-B/16 backbones, LS-B yields stable, backbone-specific allocations. On the two backbones with placement search, AugReg and iBOT, the selected blocks overlap the intermediate-depth region identified by search. Under matched storage and observation budgets, the selected blocks outperform the shallowest and deepest four-block configurations. On Split ImageNet-R, LS-B uses 60% of full-BiLoRA adapter storage while remaining within 1.5 pp of its final accuracy. The allocation requires no labels or backpropagation, adds under 0.6% runtime, and exhibits backbone-specific cortical signatures.
| Comments: | 21 pages, 12 figures |
| Subjects: | Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.01590 [cs.LG] |
| (or arXiv:2610.01590v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.01590 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuan Huang [view email]
[v1]
Thu, 1 Oct 2026 12:42:26 UTC (2,944 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org