arXiv：cs.LG· Ajay Vikram Periasami, Junlin Wang, Bhuwan Dhingra·· 13 小时前
Vision2Code：多领域图像转代码生成评测基准
Vision2Code: A Multi-Domain Benchmark for Evaluating Image-to-Code Generation
AI 导读
Vision2Code 是一个多领域、无需参考代码的图像转代码评测基准，包含来自 15 个数据集的 539 个测试样例，覆盖图表、几何、图结构、科学示意图、文档和 3D 空间场景。
正文
Abstract:We study whether vision-language models (VLMs) can reconstruct structured, graphics-based images by writing executable code to generate them. Existing benchmarks on image-to-code generation are generally limited in the domains they cover, rely on reference source code for evaluation, and do not provide metrics to quantify domain-specific issues. To address this, we introduce Vision2Code, a multi-domain, reference-code-free benchmark containing 539 test examples from 15 different datasets, spanning charts and plots, geometry, graphs, scientific diagrams, documents and 3D spatial scenes. Evaluation is done by executing the generated programs and comparing their rendered outputs to the source images using dataset- and domain-specific rubrics for a VLM judge. We find that the final scores obtained in this manner have substantially higher correlation with human ratings than those obtained via generic VLM rubrics or standard pixel-based, embedding-based or perceptual similarity metrics. We further demonstrate the reliability of our evaluation under repeated sampling from the VLM judges, varying rubric weights, and in the presence of controlled errors. Our results highlight the variability across domains of existing image-to-code generation models, with accuracy on charts and graphs being significantly higher than other domains. Lastly, we demonstrate the utility of our scoring method for rejection sampling of generated programs for fine-tuning, improving Qwen3.5-9B by 16% and Phi-4-MM by 28% compared to their base checkpoints. Our code and data are publicly available at this https URL.
| Comments: | Project page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.11307 [cs.CV] |
| (or arXiv:2605.11307v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.11307 arXiv-issued DOI via DataCite |
Submission history
From: Ajay Vikram Periasami [view email]
[v1]
Mon, 11 May 2026 22:50:12 UTC (32,276 KB)
[v2]
Fri, 9 Oct 2026 16:51:01 UTC (35,059 KB)
来源：arXiv：cs.LG · arxiv.org