为连续仿真优化设计有原则的 Expected Improvement 方法

View PDF HTML (experimental)

Abstract:Expected improvement based on stochastic kriging (SK) surrogate is widely used in continuous simulation optimization, but unknown heteroscedastic noise makes three basic design choices consequential: the incumbent to define improvement, replication allocation, and final recommendation. Existing methods typically either assume the noise levels are known or choose these components separately through heuristic rules or empirical calibration. We instead derive a high-probability upper bound on the simple regret and systematically choose the three components within general rule classes to tighten this bound. We first establish a finite-sample confidence bound for the SK predictor as a foundation for the three design choices. Within general posterior-quantile rule classes, tightening the regret certificate selects a lower-quantile incumbent and an upper-quantile recommendation among sampled designs. The opposing quantiles reflect the different roles of exploratory experiments and the final implemented decision. Under mild conditions satisfied by common kernels, minimizing a tractable kernel-specific bound yields a logarithmic replication order. Based on these design choices, the resulting simple regret matches the existing results under known homoscedastic noise up to logarithmic factors. To our knowledge, this is the first simple-regret guarantee of this form for a continuous-domain EI-type simulation optimization method that learns unknown heteroscedastic variances through replications.

Subjects:	Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as:	arXiv:2610.18434 [stat.ML]
	(or arXiv:2610.18434v2 [stat.ML] for this version)
	https://doi.org/10.48550/arXiv.2610.18434 arXiv-issued DOI via DataCite

Submission history

From: Songhao Wang [view email]
[v1] Tue, 13 Oct 2026 14:10:35 UTC (251 KB)
[v2] Wed, 14 Oct 2026 08:02:29 UTC (251 KB)