Know-Thy-Not：双编码器视觉语言模型的无训练定向否定

Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models

AI 导读

Know-Thy-Not（KTN）是一种无需训练的方法，通过引入图像侧拒绝维度，在视觉表示中直接惩罚被否定概念，解决双编码器视觉语言模型对否定句识别失败的问题。研究同时提出 COCO-Confusion 基准，首次要求模型真正拒绝被否定概念才能通过。在 9 个骨干网络上，KTN 将 COCO-Confusion mAP 较原版 VLM 最高提升 +25.6，较最强现有方法提升 +9.6。

正文

View PDF HTML (experimental)

Abstract:Dual-encoder vision-language models systematically fail at negation, scoring ''a cat without a dog'' nearly identically to ''a cat with a dog.'' We show that this failure is inherent to the single-embedding framework. Under contrastive training, single-embedding methods can at best ignore the negated concept (dilution) but never penalize its presence (rejection). We propose Know-Thy-Not (KTN), a training-free method that introduces an image-side rejection dimension to penalize the negated concept directly in the visual representation. We also introduce COCO-Confusion, the first benchmark requiring genuine rejection to succeed. Across 9 backbones, KTN improves COCO-Confusion mAP by up to +25.6 over vanilla VLMs and +9.6 over the strongest existing method.

Comments:	9 pages
Subjects:	Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.17440 [cs.CV]
	(or arXiv:2610.17440v1 [cs.CV] for this version)
	https://doi.org/10.48550/arXiv.2610.17440 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: DoYoung Kim [view email]
[v1] Mon, 12 Oct 2026 21:58:20 UTC (4,575 KB)

来源：arXiv：cs.AI · arxiv.org