arXiv：cs.AI· Slimane Larabi·· 16 小时前
超越黑箱：面向文本嵌入的视觉接地
Beyond Black Boxes: Visual Grounding for Text Embeddings
AI 导读
研究者提出一种基于草图的视觉接地框架，将文本表示转换为视觉地形草图，以呈现文本嵌入中的语义内容。在 MS-COCO 数据集上的检索评测显示，SBERT 在严格 top-1 准确率上仍占优，与草图方法在 top-5 检索上表现相当。两者级联的方法优于单独使用 SBERT，作者认为若直接在文本上训练，草图表示在信息检索上具有潜力。
正文
Abstract:In this paper, our goal is to represent text embeddings as sketches for three main reasons. First, to visualize text representation and to explore their potential their potential for AI explainability. Second, to investigate the potential benefits of the sketch-based representation for information retrieval. Third, to explore their potential for reasoning, drawing inspiration from the concept of mental imagery. We propose a sketch-based visual grounding framework that transforms text representations into visual topographic sketches. Our approach aims to emphasis the semantics content encoded in the text embeddings through the corresponding sketch.
Evaluations are conducted on the MS-COCO dataset to examine the utility of these sketches for text retrieval. The comparative study of SBERT and Sketch-based method reveals that SBERT retains an advantage in strict top-1 retrieval accuracy while achieving comparable performance to the sketch-based method in top-5 retrieval. A cascaded approach combining SBERT and the sketch-based method outperforms SBERT alone, suggesting that the Sketch-based representation could offer significant potential for information retrieval if trained directly on text.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18047 [cs.AI] |
| (or arXiv:2610.18047v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18047 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Slimane Larabi [view email]
[v1]
Tue, 13 Oct 2026 09:37:27 UTC (1,648 KB)
来源：arXiv：cs.AI · arxiv.org