从实证评估到上下文增强：用 LLM 修复回归错误

View PDF HTML (experimental)

Abstract:[...] Since then, various APR approaches, especially those leveraging the power of large language models (LLMs), have been rapidly developed to fix general software bugs. Unfortunately, the effectiveness of these advanced techniques in the context of regression bugs remains largely unexplored. This gap motivates the need for an empirical study evaluating the effectiveness of modern APR techniques in fixing real-world regression bugs.
In this work, we conduct an empirical study of APR techniques on regression bugs. To facilitate our study, we introduce RegressionBug4APR, a high-quality benchmark of Java and Python regression bugs integrated into a framework designed to facilitate APR research. The current benchmark includes 200 regression bugs collected from widely used real-world GitHub repositories. We begin by conducting an in-depth analysis of the benchmark, demonstrating its diversity and quality. Building on this foundation, we empirically evaluate the capabilities of APR to regression bugs by assessing both traditional APR tools and advanced LLM-based APR approaches. Our experimental results show that classical APR tools fail to repair any bugs, while LLM-based APR approaches exhibit promising potential. Motivated by these results, we investigate impact of incorporating bug-inducing change information into LLM-based APR approaches for fixing regression bugs. We further conduct an ablation study to disaggregate the contribution of each contextual element within the bug-inducing change information. Our results highlight that this context-aware enhancement significantly improves the performance of LLM-based APR, yielding 1.6x more successful repairs compared to using LLM-based APR without such context. Moreover, our findings are consistent across both Java and Python benchmarks, providing preliminary evidence for the generalizability of our findings.

Comments:	This paper has been peer-reviewed and accepted for publication in ACM Transactions on Software Engineering and Methodology (TOSEM)
Subjects:	Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2506.13182 [cs.SE]
	(or arXiv:2506.13182v4 [cs.SE] for this version)
	https://doi.org/10.48550/arXiv.2506.13182 arXiv-issued DOI via DataCite

Submission history

From: Anh Ho [view email]
[v1] Mon, 16 Jun 2025 07:49:18 UTC (2,389 KB)
[v2] Sat, 20 Jun 2026 16:37:07 UTC (2,735 KB)
[v3] Tue, 25 Aug 2026 01:22:56 UTC (2,704 KB)
[v4] Tue, 13 Oct 2026 03:32:20 UTC (2,660 KB)