Targeted Variance Reduction: Effective Bayesian Optimization of Black-Box Simulators with Noise Parameters

The optimization of a black-box simulator over control parameters (Formula presented.) arises in a myriad of scientific applications. In such applications, the simulator often takes the form (Formula presented.), where (Formula presented.) are parameters that are uncertain in practice. Stochastic optimization aims to optimize the objective (Formula presented.), where (Formula presented.) is a random variable that models uncertainty on (Formula presented.). For this, existing black-box methods typically employ a two-stage approach for selecting the next point (Formula presented.), where (Formula presented.) and (Formula presented.) are optimized separately via different acquisition functions. As such, these approaches do not employ a joint acquisition over (Formula presented.), and thus may fail to fully exploit control-to-noise interactions for effective stochastic optimization. To address this, we propose a new Bayesian optimization method called Targeted Variance Reduction (TVR). The TVR leverages a novel joint acquisition function over (Formula presented.), which targets variance reduction on the objective within the desired region of improvement. Under a Gaussian process surrogate with a squared exponential covariance function on f, the TVR acquisition can be evaluated in closed form, and reveals an insightful exploration-exploitation-precision tradeoff for stochastic black-box optimization. The TVR can further accommodate a broad class of non-Gaussian distributions on (Formula presented.) via a careful integration of normalizing flows. We demonstrate the improved performance of TVR over the state-of-the-art in a suite of numerical experiments and an application to the robust design of automobile brake discs under operational uncertainty.

Duke Scholars

Author Simon Mak Statistical Science