arXiv：cs.AI· Youmin Ko, Hyemin Yang, Taeuk Kim, Hyunjoon Kim·· 16 小时前
SATKGC：面向知识图谱补全的子图感知语言模型训练框架
Subgraph-Aware Training of Language Models for Knowledge Graph Completion Using Structure-Aware Contrastive Learning
AI 导读
研究者提出子图感知训练框架 SATKGC，通过子图感知 mini-batching 与结构感知对比学习，将知识图谱的结构归纳偏置引入 PLM 微调，以提升知识图谱补全准确率。该工作称首次系统性地把子图、最短路径、度数等结构信息融入 PLM 微调，在三个 KGC benchmark 上验证了其优越性，论文已被 The Web Conference 2025 接收。
正文
Abstract:Fine-tuning pre-trained language models (PLMs) has recently shown a potential to improve knowledge graph completion (KGC). However, most PLM-based methods focus solely on encoding textual information, neglecting the long-tailed nature of knowledge graphs and their various topological structures, e.g., subgraphs, shortest paths, and degrees. We claim that this is a major obstacle to achieving higher accuracy of PLMs for KGC. To this end, we propose a Subgraph-Aware Training framework for KGC (SATKGC) with two ideas: (i) subgraph-aware mini-batching to encourage hard negative sampling and to mitigate an imbalance in the frequency of entity occurrences during training, and (ii) new contrastive learning to focus more on harder in-batch negative triples and harder positive triples in terms of the structural properties of the knowledge graph. To the best of our knowledge, this is the first study to comprehensively incorporate the structural inductive bias of the knowledge graph into fine-tuning PLMs. Extensive experiments on three KGC benchmarks demonstrate the superiority of SATKGC. Our code is available.
| Comments: | Accepted to The Web Conference 2025 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2407.12703 [cs.CL] |
| (or arXiv:2407.12703v5 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2407.12703 arXiv-issued DOI via DataCite |
Submission history
From: Youmin Ko [view email]
[v1]
Wed, 17 Jul 2024 16:25:37 UTC (8,250 KB)
[v2]
Fri, 19 Jul 2024 00:34:13 UTC (8,250 KB)
[v3]
Tue, 23 Jul 2024 06:26:30 UTC (8,250 KB)
[v4]
Tue, 15 Oct 2024 15:59:09 UTC (4,180 KB)
[v5]
Fri, 31 Jan 2025 12:56:16 UTC (1,626 KB)
来源：arXiv：cs.AI · arxiv.org