arXiv：cs.LG· Vincent Szolnoky·· 4 小时前
D-SLR：不相交行稀疏加低秩分解，截断 SVD 的闭式替代方案
D-SLR: The Disjoint Row-Sparse plus Low-Rank Decomposition
AI 导读
D-SLR 是一种不相交行稀疏加低秩分解，作为截断 SVD 的闭式替代方案，在平方误差下可达到联合最优，且在每个非平凡秩与存储行数下参数更少。该方法仅在零存储行时退化为截断 SVD，因此同等成本下不会更差；整个误差—参数权衡网格与求解仅需三次 SVD，无需调参或正则化。研究还给出每个形状下无假设的误差下界证书，并在 LLM 嵌入表、网络流量和高光谱图像等合成与真实数据上验证了增益。
正文
Abstract:Compressing a matrix for reconstruction still defaults to the truncated SVD, approximating the data with a single low-rank structure. It is common to reduce the residual further by adding an overlapping row-sparse component, but methods that solve this joint problem often require iterative solvers and tuning of regularization parameters. We propose the Disjoint Row-Sparse plus Low-Rank (D-SLR) decomposition, a closed-form drop-in for the truncated SVD that improves or exactly matches it. D-SLR restricts rows to either being stored verbatim or approximated by the low-rank fit, never both. Under squared error this restriction costs nothing: the joint optimum is attainable disjointly with fewer parameters at every non-trivial rank and stored row count (shape). With zero stored rows D-SLR reduces to the truncated SVD, so it never does worse at equal cost. The algorithm scores the entire error-versus-parameters tradeoff, and the solution is chosen afterwards by a supplied error target or parameter count, or by a selection rule. The grid and solution together cost three SVDs, with no tuning or regularization. We derive an assumption-free, a-posteriori lower bound on the error at every shape, giving each solution a computable certificate on the potential gain of any other choice of rank and stored rows. Experiments on synthetic and real data (LLM embedding tables, network traffic, hyperspectral images) confirm the gains and quantify the certificate.
| Subjects: | Machine Learning (cs.LG); Methodology (stat.ME); Machine Learning (stat.ML) |
| Cite as: | arXiv:2610.10636 [cs.LG] |
| (or arXiv:2610.10636v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10636 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vincent Szolnoky [view email]
[v1]
Wed, 7 Oct 2026 14:07:41 UTC (139 KB)
来源：arXiv：cs.LG · arxiv.org