arXiv：cs.LG· Hengguang Zhou, Yuanhao Ban, Jiaqi Feng, Xiaohuan Pei, Justin Cui, Cho-Jui Hsieh·· 4 小时前
编程智能体在视觉生成上能走多远？基于 AgentDrawBench 的研究
How Far Can Coding Agents Go in Visual Generation?
AI 导读
研究用 200 条提示词的 AgentDrawBench 测试编程智能体的零样本绘图能力，发现它们擅长保持结构和满足精确视觉约束，在构图准确性上表现有竞争力，但感知丰富度有限。视觉表达力随绘图接口变化，重建的视觉代码在视觉问答任务中保留大量信息，部分领域准确率高于原图。跨绘图框架和独立单像素查询生成的图像呈现相近布局与色调，且在可见绘图移除后仍能保持自生成的房间布局。
正文
Abstract:Modern image generation largely relies on specialized generative models that leave the process of image construction implicit in learned parameters. We explore an alternative: general-purpose coding agents that construct images through executable programs. Our study of zero-shot drawing on AgentDrawBench, a 200-prompt benchmark, reveals a distinctive capability profile. Agents excel at preserving structure and satisfying precise visual constraints, achieving competitive results in domains where compositional accuracy matters more than photorealistic appearance, while remaining limited in perceptual richness. Their visual expressiveness varies with the drawing interface, and a qualitative study on reconstruction shows how procedural skills extracted from human demonstrations support different artistic media. We then examine code as an external visual language of thought: a structured, executable representation that agents can construct and inspect. We test it in visual question answering task and find reconstructed visual code retains substantial information for question answering and, in some domains, supports higher accuracy than the original image. These abilities raise a further question: do agents draw on a shared internal representation of scene content beyond a particular drawing program? We explore an internal visual language-of-thought hypothesis, in which a shared, compositionally structured representation supports visual operations across drawing languages. Cross-harness drawings and images assembled from independent single-pixel queries exhibit related layouts and tones. Agents also maintain self-generated room arrangements across viewpoints after visible drawings are removed. These results demonstrate that the coding agent would maintain a consistent and coherent world inside its imagine even before the harness gets involved and the generation starts.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.19025 [cs.CV] |
| (or arXiv:2610.19025v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19025 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hengguang Zhou [view email]
[v1]
Tue, 13 Oct 2026 18:18:41 UTC (6,216 KB)
来源：arXiv：cs.LG · arxiv.org