Gpyopt: How to save and load

[YoshikawaMasashi]

Dear SheffieldML

It is very convenient to use Bayesian optimization in Python.

I want to preserve the optimization halfway, load it later and resume optimization.
Is there a way to do this?

[javiergonzalezh]

Hi,

These are the two options that you have:

1) You can simply re run object.run_optimization(iterations) as many times as you want so yo always resume the optimization.

2) You can save the parameters of all your models and the collected data in a tex file. GPyOpt can do that for you. If you want to re-start from some specific point you'll need the create a wrapper to initialize another GPyOpt object using the data in the files. Unfortunately, that is not implemented but if you want to give it a try it would be great! :)

Javier

[YoshikawaMasashi]

Dear Javier

Thank you for the information very much.

I saved X and Y, and it worked.

[javiergonzalezh]

Great!

[iantheconway]

Hey, I'm trying to save and load a model as I perform optimization so that work is always cached, and I was wondering if the way I'm doing it works:

max_iter = 100
myProblem = GPyOpt.methods.BayesianOptimization(gpy_helper, bounds)
myProblem.run_optimization(1)
myProblem.save_evaluations("ev_file")
for i in range(max_iter - 1):
    evals = pd.read_csv("ev_file", index_col=0, delimiter="\t")
    Y = np.array([[x] for x in evals["Y"]])
    X = np.array(evals.filter(regex="var*"))
    myProblem = GPyOpt.methods.BayesianOptimization(gpy_helper, bounds, X=X, Y=Y)
    myProblem.run_optimization(1)
    myProblem.save_evaluations("ev_file")

[xnchu]

I wonder how to use previously generated X and Y to create new GPyOpt? Is there any example or tutorials? I asked this question because my functions (neural networks) take a long time to run, and whenever GPyOpt stops, I have to re-run the whole process. How to implement the ' create a wrapper to initialize another GPyOpt object using the data in the files. ' as mentioned by javiergonzalezh. Anyone can help please?

[iantheconway]

@xnchu See my example above, I think it does what you want. It assumes that you gpy_helper is an objective function in the proper format for GPyOpt. If you want to load an experiment that you've already run, you can skip the initialization and just run the loop:
max_iter = 100 for i in range(max_iter - 1): evals = pd.read_csv("ev_file", index_col=0, delimiter="\t") Y = np.array([[x] for x in evals["Y"]]) X = np.array(evals.filter(regex="var*")) myProblem = GPyOpt.methods.BayesianOptimization(gpy_helper, bounds, X=X, Y=Y) myProblem.run_optimization(1) myProblem.save_evaluations("ev_file")
Assuming that ev_file (or whatever you choose to call it) has already been created by calling myProblem.save_evaluations("ev_file") after your bayes opt.

[shaielkayam]

Hi, can some one explain me what is the differrence between runing my the function 10 times with:
opt_model = GPyOpt.methods.BayesianOptimization(f=f, domain=bounds, initial_design_numdata=0)
opt_model.run_optimization(max_iter=10, max_time=72000)
and running it 10 times with:
opt_model = GPyOpt.methods.BayesianOptimization(f=f, domain=bounds, initial_design_numdata=5)
opt_model.run_optimization(max_iter=5, max_time=72000)

seems that in both ways, the BayesianOptimization just run the function f the same way. why should i use initial_design_numdata different then 0?