Gpyopt: Can GPyOpt suggest next round of experiments?

[lewisscola]

Hi,
I want to explain my problem with the example in the website: https://www.blopig.com/blog/wp-content/uploads/2019/10/GPyOpt-Tutorial1.html
The codes are:
import GPyOpt
from GPyOpt.methods import BayesianOptimization
import numpy as np
from numpy.random import multivariate_normal #For later example

import pandas as pd
from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
import numpy as np
from numpy.random import multivariate_normal

def obj_func_2d(x,y):
return((x2 + y2)(np.sin(x)*2 - np.cos(y)))

fig = plt.figure(figsize=plt.figaspect(0.3))
fig.suptitle('Plots of our objective function')

X = np.arange(0, 10, 0.25)
Y = np.arange(0, 10, 0.25)
X, Y = np.meshgrid(X, Y)
Z = obj_func_2d(X,Y)

ax = fig.add_subplot(1, 2, 1)
ax.contour(X,Y,Z)

ax = fig.add_subplot(1, 2, 2, projection='3d')

surf = ax.plot_surface(X, Y, Z, cmap=cm.coolwarm,
linewidth=0, antialiased=False)

ax.zaxis.set_major_locator(LinearLocator(10))
ax.zaxis.set_major_formatter(FormatStrFormatter('%.02f'))

fig.colorbar(surf, shrink=0.5, aspect=5)

plt.show()

def objfunc2d(x):
"""
x is a 2 dimensional vector.
"""
x1 = x[:, 0]
x2 = x[:, 1]
return((x12 + x22)(np.sin(x1)*2 - np.cos(x2)))

bounds2d = [{'name': 'var_1', 'type': 'continuous', 'domain': (0,10)},
{'name': 'var_2', 'type': 'continuous', 'domain': (0,10)}]
maxiter = 50

myBopt_2d = GPyOpt.methods.BayesianOptimization(objfunc2d, domain=bounds2d)
myBopt_2d.run_optimization(max_iter = maxiter)
print("="20)
print("Value of (x,y) that minimises the objective:"+str(myBopt_2d.x_opt))
print("Minimum value of the objective: "+str(myBopt_2d.fx_opt))
print("="20)
myBopt_2d.plot_acquisition()

My question is:
According to the above Bayesian Optimization(BO) results, we can find the optimal of the function.
However, if I want to update the surrogate model with actual experiments(for example, add some points into the model, and then redo BO), can I get these suggested points from the results of BO by using the module in GPyOpt?

Thank you for your help!

[ekalosak]

Here's the tutorial for using an external objective function. Specifically, you might be interested in this chunk:

while current_iter < iter_count:
    bo_step = GPyOpt.methods.BayesianOptimization(f = None, domain = domain, X = X_step, Y = Y_step)
    x_next = bo_step.suggest_next_locations()
    y_next = func.f(x_next)

    X_step = np.vstack((X_step, x_next))
    Y_step = np.vstack((Y_step, y_next))

    current_iter += 1

For reference, tutorials for common problems are available here.

[lewisscola]

Thank you very much for your reply!
I have another question about 'suggest_next_locations()' function in the above example.
If I want to have multiple next locations, can I use 'batch size' to realize it?

For example, I simply write: batch size= 4

Could you show me how to use it if I want to get 4 suggested locations for next experiments.

[apaleyes]

Prettu sure "batch_size" refers to local penalization, the only acquisition function currently implemented in GPyOpt that actually supports batches.

Alas, our API isn't clear enough to make users aware that this parameter is meaningless with other acquisition functions.

[lewisscola]

Hi,
Thank you for your explanation!
If I want to run multiple Bayesian Optimizations in the above examples( for example 5 runs), is it possible to realize it in GpyOpt?
For instance, I run 5 Bayesian Optimizations(based on the same model) and then I can calculate the average performance of BO so that I can know the variance during BO. As you can see the below picture(I can plot the error bar along with the average performance line).

[michaeljeaffresoncass]

Prettu sure "batch_size" refers to local penalization, the only acquisition function currently implemented in GPyOpt that actually supports batches.

Alas, our API isn't clear enough to make users aware that this parameter is meaningless with other acquisition functions.

I am not sure I follow this - I was going to ask about generate a set of multiple next suggestions as is indicated by the quoted documentation adding batch size parameter I would expect to do this but doesn't actually do anything. It sounds like if I do experiments in batches of say 5 there is no current way to ask for suggests 5 at a time - I can probably work out a work around but given experiments are often done in batches and you have a function that says next suggests (plural) it might be a nice one to implement. I have used other optimisers where you can request a number of exploitation and a number of exploration suggestions. I guess this need digging into the acquisition function and the surrogate some.