arXiv：cs.AI· Jiaxin Fang, Runyuan He, Arnav Gupta, Sahil Kumawat, Huanzhi Mao, Alvin Cheung·· 1 天前
CodeWalk：用问题图游走自动生成编程基准测试
CodeWalk: Generating Coding Benchmarks by Walking a Problem Graph
AI 导读
CodeWalk 通过组合已验证的原子编程题生成 level-N 新任务，整个过程程序化完成，无需语言模型生成或验证。该方法覆盖多 bug 修复、证明补全、复杂度分析和克隆等价四类场景，前沿模型随着组合深度增加表现持续下降；用其做拒绝采样微调还能逐级提升 Qwen3.5-9B。
正文
Abstract:High-quality coding problems are expensive to collect. Benchmarks such as SWE-bench rely on substantial human curation, while synthetic alternatives typically rely on costly model generation and validation. In this work, we show that new coding problems can be created at almost no cost by composing existing ones. We formulate problem generation as code composition: verified atomic problems are combined into level-$N$ tasks, and a task is kept only if every atomic problem remains unsolved and a joint reference solution passes the same check used at evaluation time. The entire task-construction process is programmatic and requires no language-model generation or validation. We instantiate this approach in four coding settings: multi-bug repair, proof completion, complexity analysis, and clone equivalence. The generated tasks are challenging for frontier models, which all degrade as compositional depth increases. The generated tasks are also useful as training data: rejection-sampling fine-tuning on them improves Qwen3.5-9B at every composition level.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.15345 [cs.LG] |
| (or arXiv:2610.15345v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.15345 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiaxin Fang [view email]
[v1]
Sun, 11 Oct 2026 04:57:00 UTC (251 KB)
来源：arXiv：cs.AI · arxiv.org