Robust Gaussian process-based global optimization using a fully Bayesian expected improvement criterion

Abstract : We consider the problem of optimizing a real-valued continuous function f, which is supposed to be expensive to evaluate and, consequently, can only be evaluated a limited number of times. This article focuses on the Bayesian approach to this problem, which consists in combining evaluation results and prior information about f in order to efficiently select new evaluation points, as long as the budget for evaluations is not exhausted. The algorithm called efficient global optimization (EGO), proposed by Jones, Schonlau and Welch (J. Global Optim., 13(4):455-492, 1998), is one of the most popular Bayesian optimization algorithms. It is based on a sampling criterion called the expected improvement (EI), which assumes a Gaussian process prior about f. In the EGO algorithm, the parameters of the covariance of the Gaussian process are estimated from the evaluation results by maximum likelihood, and these parameters are then plugged in the EI sampling criterion. However, it is well-known that this plug-in strategy can lead to very disappointing results when the evaluation results do not carry enough information about f to estimate the parameters in a satisfactory manner. We advocate a fully Bayesian approach to this problem, and derive an analytical expression for the EI criterion in the case of Student predictive distributions. Numerical experiments show that the fully Bayesian approach makes EI-based optimization more robust while maintaining an average loss similar to that of the EGO algorithm.
Complete list of metadatas

Cited literature [32 references]  Display  Hide  Download

https://hal-supelec.archives-ouvertes.fr/hal-00607816
Contributor : Karine El Rassi <>
Submitted on : Monday, July 11, 2011 - 2:27:37 PM
Last modification on : Thursday, March 29, 2018 - 11:06:04 AM
Long-term archiving on : Monday, November 12, 2012 - 10:41:31 AM

File

Lion_5_paper.pdf
Files produced by the author(s)

Identifiers

Collections

Citation

Romain Benassi, Julien Bect, Emmanuel Vazquez. Robust Gaussian process-based global optimization using a fully Bayesian expected improvement criterion. 5th International Conference on Learning and Intelligent Optimization (LION 5), Jan 2011, Rome, Italy. pp.176-190, ⟨10.1007/978-3-642-25566-3_13⟩. ⟨hal-00607816⟩

Share

Metrics

Record views

562

Files downloads

926