arXiv：cs.LG· Songyuan Sui, Srikanth Malla, Chiho Choi, Joon Hee Choi·· 3 小时前AI 评分45
每个用户都需要私有 LoRA 吗？LINEUP 将个性化与逐用户适配解耦
Does Every User Need a Private LoRA? Decoupling Personalization from Per-User Adaptation
AI 导读
研究提出 LINEUP，通过学习一组可复用的低秩个性化因子，经用户条件召回与查询相关校准组合，把目标用户适配限制在共享修正空间上的极小用户代码中，每个用户只需优化 8 个标量，而对比的私有 LoRA 配置每用户需 419 万参数。
正文
Abstract:Personalized large language models often require a complete adaptation state for each user. However, this paradigm scales poorly as the user population grows. We revisit this design through the lens of personalization capacity allocation: how much adaptation capacity can be shared across users, how the shared capacity should be composed, and how much must remain user-specific. We answer them through three complementary empirical analyses. We find that independent user adapters contain substantial cross-user reusable structure, that the utility of reusable directions reflects both user relevance and variation across queries, and that user histories provide transferable signals for compact individual correction. Motivated by these findings, we propose LINEUP. It learns a bank of reusable low-rank personalization factors, composes them through user-conditioned recall and query-dependent calibration, and restricts target-user adaptation to a tiny user code over a shared correction space. This design decouples expressive personalization capacity from per-user trainable state. Each target user optimizes only eight scalars, while all shared components remain fixed. By comparison, the evaluated private-LoRA configuration uses 4.19 million per-user parameters. Our theoretical analysis gives a finite-step, finite-history risk bound and sufficient conditions for user-code refinement to improve on history initialization. Across six tasks spanning personalized classification, prediction, and generation, LINEUP leads on all 12 metrics, each averaged over three independent runs (e.g., reducing LaMP-3 RMSE by 11.4% relative to the strongest baseline). It maintains advantages under limited history. These results show that rich personalization can be supported primarily by reusable, conditionally composed shared capacity, while independent user adaptation remains confined to a tiny correction state.
| Comments: | 10 pages main content, 36 pages total including appendix, 7 figures |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.02353 [cs.LG] |
| (or arXiv:2610.02353v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.02353 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Songyuan Sui [view email]
[v1]
Thu, 1 Oct 2026 18:29:36 UTC (396 KB)
来源：arXiv：cs.LG · arxiv.org