Hyper-parameter handling for Gaussian processes in efficient global optimization

In simulation-based optimization, a common issue with many meta-heuristic algorithms is the limited computational budget. Performing a simulation is usually considerably more time-consuming than evaluating a closed mathematical function. Surrogate-assisted algorithms alleviate this problem by using representative models of the simulation which can be evaluated much faster. One of the most promising surrogate-assisted optimization approaches is Efficient Global Optimization, which uses Gaussian processes as surrogate-models. In this paper, the importance of carefully chosen hyper-parameters for Gaussian process kernels and a way of self-configuration is shown. Based on properties of the training set, e.g. distances between observed points, observed target values, etc., the hyper-parameters of the used kernels are initialized and bounded accordingly. With these initial values and bounds in mind, hyper-parameters are then optimized, which results in improved Gaussian process models that can be used for regression. The goal is to provide an automated way of hyper-parameter initialization, which can be used when building Kriging models in surrogate-assisted algorithms, e.g. Efficient Global Optimization (EGO). Obtained results show that applying the proposed hyper-parameter initialization and bounding can increase the performance of EGO in terms of either convergence speed or achieved objective function value.