DBA-Bench：面向 LLM 数据库运维智能体的生产级保真基准

View PDF HTML (experimental)

Abstract:LLM-based database agents show promise, but differences in task scope, testbeds, and success criteria hinder direct comparison. A shared benchmark must also preserve the operational conditions that determine whether diagnosis and remediation succeed in practice. We present DBA-Bench, guided by operational realism, outcome-first evaluation, and controlled scenario reproducibility. It evaluates agents in live PostgreSQL and distributed TiDB deployments with active workloads, stateful behavior, and multi-source observations. Success requires satisfying scenario-specific operational objectives without violating safety constraints. The primary evaluation covers 231 bounded scenarios across five operational domains, while 10 persistent scenarios extend evaluation to continued operation across interacting events. We evaluate eight automated baselines, including four frontier foundation models, two open-weight models, and two GPT-5.5-backed database agents. Our evaluation exposes gaps between diagnosis, effective remediation, and safe execution across both database systems. Agents can identify root causes yet fail to restore operational objectives. Even when objectives are restored, the executed actions can violate safety constraints. Persistent scenarios expose a gap between clearing individual blockers and sustaining business service: agents leave competing workloads uncoordinated and previously completed repairs unchecked after subsequent changes.

Comments:	15 pages, 9 figures, 4 tables
Subjects:	Databases (cs.DB); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as:	arXiv:2607.22165 [cs.DB]
	(or arXiv:2607.22165v2 [cs.DB] for this version)
	https://doi.org/10.48550/arXiv.2607.22165 arXiv-issued DOI via DataCite

Submission history

From: Junming Chen [view email]
[v1] Fri, 24 Jul 2026 10:11:24 UTC (1,135 KB)
[v2] Sat, 10 Oct 2026 10:57:03 UTC (2,416 KB)