Gpyopt: IndexError: bug or problem?

[1Reinier]

Hi there,

I can't seem to figure out the cause of the following problem (stacktrace included below).

My problem is the optimization of an objective function with six hyperparameters, of which two are discrete and the rest continuous.

Could you help me out where I made a mistake, or where the bug is?

Code:

def objective(args):
    arg = args.reshape(-1,)
    # ...
    return np.array([[mse]])

domain = [
# ...
]

initial_parameters = np.array([[1000, 0.5, 0.3, 1.25, 1e-8, 10], 
                               [1000, 0.5, 0.3, 1.25, 1e-8, 10], 
                               [1000, 0.5, 0.3, 1.25, 1e-8, 10]])

initial_mse = np.array([[0.15945], 
                        [0.10871], 
                        [0.053224]])

bo = GPyOpt.methods.BayesianOptimization(f=objective, domain=domain, 
X=initial_parameters, Y=initial_mse) 

Stacktrace:

IndexError                                Traceback (most recent call last)
<ipython-input-17-ca9b66ea1c32> in <module>()
---> 31 bo = GPyOpt.methods.BayesianOptimization(f=objective, domain=domain, X=initial_parameters, Y=initial_mse)
     32 
     33 

/Users/Reinier/anaconda/lib/python3.5/site-packages/GPyOpt/methods/bayesian_optimization.py in __init__(self, f, domain, constrains, cost_withGradients, model_type, X, Y, initial_design_numdata, initial_design_type, acquisition_type, normalize_Y, exact_feval, acquisition_optimizer_type, model_update_interval, evaluator_type, batch_size, num_cores, verbosity, verbosity_model, bounds, **kwargs)
    234 
    235         # --- Initialize everything
--> 236         self.run_optimization(max_iter=0,verbosity=self.verbosity)
    237 
    238     def _model_chooser(self):

/Users/Reinier/anaconda/lib/python3.5/site-packages/GPyOpt/methods/bayesian_optimization.py in run_optimization(self, max_iter, max_time, eps, verbosity, save_models_parameters, report_file, evaluations_file, models_file, **kwargs)
    450                 self.acquisition_optimizer.optimizer ='CMA'
    451             print('WARNING: "acqu_optimize_method" will be deprecated in the next version!')
--> 452         super(BayesianOptimization, self).run_optimization(max_iter = max_iter, max_time = max_time,  eps = eps, verbosity=verbosity, save_models_parameters = save_models_parameters, report_file = report_file, evaluations_file= evaluations_file, models_file=models_file)
    453 
    454 

/Users/Reinier/anaconda/lib/python3.5/site-packages/GPyOpt/core/bo.py in run_optimization(self, max_iter, max_time, eps, verbosity, save_models_parameters, report_file, evaluations_file, models_file)
    104 
    105             # --- Update and optimize acquisition and compute the exploration level in the next evaluation
--> 106             self.suggested_sample = self._compute_next_evaluations()
    107 
    108             if not ((self.num_acquisitions < self.max_iter) and (self._distance_last_evaluations() > self.eps)):

/Users/Reinier/anaconda/lib/python3.5/site-packages/GPyOpt/core/bo.py in _compute_next_evaluations(self)
    182         Computes the location of the new evaluation (optimizes the acquisition in the standard case).
    183         """
--> 184         return self.evaluator.compute_batch()
    185 
    186     def _update_model(self):

/Users/Reinier/anaconda/lib/python3.5/site-packages/GPyOpt/core/evaluators/sequential.py in compute_batch(self)
     19         Selects the new location to evaluate the objective.
     20         """
---> 21         return self.acquisition.optimize()
     22 
     23 

/Users/Reinier/anaconda/lib/python3.5/site-packages/GPyOpt/acquisitions/base.py in optimize(self)
     57             out = self.optimizer.optimize(f=self.acquisition_function)[0]
     58         else:
---> 59             out = self.optimizer.optimize(f=self.acquisition_function, f_df=self.acquisition_function_withGradients)[0]
     60         return out
     61 

/Users/Reinier/anaconda/lib/python3.5/site-packages/GPyOpt/optimization/acquisition_optimizer.py in optimize(self, f, df, f_df)
    287 
    288         for i in range(num_discrete):
--> 289             self.mixed_optimizer.fix_dimensions(dims=self.discrete_dims, values=self.discrete_values[i,:])
    290             partial_x_min[i,:] , partial_f_min[i,:] = self.mixed_optimizer.optimize(f, df, f_df)
    291 

/Users/Reinier/anaconda/lib/python3.5/site-packages/GPyOpt/optimization/acquisition_optimizer.py in fix_dimensions(self, dims, values)
    118 
    119         # -- take only the fixed components of the random samples
--> 120         self.samples = self.samples[:,np.array(self.free_dims)] # take only the component of active dims
    121         self.subspace = self.space.get_subspace(self.free_dims)
    122         self.optimizer = select_optimizer(self.optimizer_name)(Design_space(self.subspace), **self.kwargs)

IndexError: index 4 is out of bounds for axis 1 with size 4

[javiergonzalezh]

Sorry for the slow response. I have been on holidays. Due to the indexing, it only seems to work well when you have all the discrete variables at the end. I am trying to fix this issue but that should do the right thing.

[1Reinier]

Thanks! If I do change the order, will it consider the argument names automatically, or do I need to change the ordering in my objective function?

[javiergonzalezh]

You need to change the order in the objective function. I need to improve this bit, it doesn't make much sense requiring an specific ordering in the input variables.

[javiergonzalezh]

The problem has been solved now in the devel branch. Have a look to it!