arXiv：cs.LG· Xinjie Zeng, Qinghua Tao, Johan Suykens·· 3 小时前AI 评分31
核奇异值分解 KSVD 扩展至多数据源：eKSVD
Kernel Singular Value Decomposition with Extension to Multiple Data Sources
AI 导读
研究者将核奇异值分解（KSVD）扩展至多数据源，提出 eKSVD，可对非对称核进行联合非线性特征学习。该方法在原问题中联合学习各数据源的投影并引入成对耦合，借助拉格朗日乘子与 KKT 条件，在對偶问题中得到 KSVD 移位特征值问题的推广，并推导出基于协方差的框架，同时用神经网络实现显式特征映射。数值实验验证了 eKSVD 处理多数据源的效果，以及神经网络为核方法带来的灵活性。
正文
Abstract:Kernel Singular Value Decomposition (KSVD) learns a pair of singular vectors w.r.t. an asymmetric kernel matrix, which can be induced by two data sources, e.g., the queries and keys in self-attention or the rows and columns of a given matrix. In this work, we extend KSVD to multiple data sources, namely eKSVD, which conducts joint nonlinear feature learning upon asymmetric kernels. In the primal formulation, the projections associated with each data source are jointly learned to capture maximal information, while incorporating pair-wise couplings. With the Lagrangian and its Karush-Kuhn-Tucker (KKT) conditions, the optimization in the dual leads to a generalization of the shifted eigenvalue problem in Lanczos decomposition theorem of KSVD. Further, a covariance-based framework is derived together with using neural networks (NNs) for explicit feature mappings, complementary to the kernel-based interpretation and optimization. Numerical experiments verify the effectiveness of our eKSVD compared to methods based on Mercer kernels for tackling multiple data sources, and our innovation of deploying NNs demonstrates great flexibility for kernel methods.
| Subjects: | Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.03216 [cs.LG] |
| (or arXiv:2610.03216v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03216 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xinjie Zeng [view email]
[v1]
Fri, 2 Oct 2026 12:32:21 UTC (133 KB)
来源：arXiv：cs.LG · arxiv.org