arXiv：cs.LG（机器学习，全量分类）· Yuzhou Wang, Emile Anand, Ijay Narang·· 15 小时前AI 评分41
Hob-VL：面向视觉基础布尔推理的基准测试
Hob-VL: A Benchmark for Visually Grounded Boolean Reasoning
AI 导读
研究者推出 Hob-VL 基准，用于评估视觉基础布尔推理，包含 6000 道人工核验的 Yes/No 题和 1000 道物体识别题，覆盖 1000 个生成场景与 46 张标注照片。在八种关闭或最小化思考的模型配置下，布尔准确率仅 48.52%–50.57%，识别准确率最高 43.0%；启用思考的 GLM 配置提升不均且仍存在大量错误与不一致。
正文
Abstract:Reliable visual reasoning requires composing multiple visual observations and returning consistent answers to logically equivalent questions. We introduce Hob-VL, a benchmark for visually grounded Boolean reasoning. Hob-VL comprises two tasks: (1) evaluating whether a Boolean rule holds in an image, and (2) identifying the (unique) object satisfying a Boolean description. Hob-VL contains 6,000 human-verified balanced Yes/No questions, each defined by a Boolean combination of ten visual statements, across 1,000 generated scenes and 46 diverse labeled photographs, along with 1,000 object-identification questions over the same photographs. Our question families are deliberately constructed to challenge reasoning through misleading local cues and nested logical operations, and include symbolic and structured natural-language presentations. Across eight model configurations with thinking disabled or minimized, Boolean accuracy ranges from 48.52% to 50.57%, while the identification accuracy reaches at most 43.0%. A thinking-enabled GLM configuration achieves uneven gains while retaining substantial errors and inconsistencies. Hob-VL exposes these failures through executable reference answers and matched evaluations.
| Comments: | 29 pages, 6 figures, 14 tables |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Logic in Computer Science (cs.LO) |
| ACM classes: | I.4.8; I.4.7; I.1.0; I.1.1; I.2.7; I.2.10 |
| Cite as: | arXiv:2610.01605 [cs.CV] |
| (or arXiv:2610.01605v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.01605 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Emile Timothy Anand [view email]
[v1]
Thu, 1 Oct 2026 12:48:54 UTC (2,876 KB)
来源：arXiv：cs.LG（机器学习，全量分类） · arxiv.org