Scikit-optimize: Request: Make objective function with kwarg parameters (instead of list)

[jolespin]

It would be nice to have a general use case objective function generator where you give it a scorer and a parameter space.

If space could be something like this:

`space = {"max_depth":(1, 5), "max_features":(1, X.shape[1]), "min_samples_split":(2, 100)}`

With a make objective class:

def make_objective(self, params):
    kwargs = dict(zip(self.param_labels, params))    
    self.model.set_params(**kwargs)
    score = np.mean(model_selection.cross_val_score(self.model, self.X, self.y, cv=self.cv, n_jobs=-1, scoring=self.scorer))
    return score

Let me know if I should elaborate on this. Basically something that makes an objective function so one can give it a scorer and parameter space in the form of a dictionary.

[MechCoder]

That was proposed here (https://github.com/scikit-optimize/scikit-optimize/pull/198)

An alternative way is to have an attribute for Dimension that has a name attribute (https://github.com/scikit-optimize/scikit-optimize/issues/424) which is more verbose than the above proposed dict-space.

[betatim]

Do you think this would be generally useful for applications beyond scikit-learn?

We recently added BayesSearchCV which has the same interface as GridSearchCV in scikit-learn. This should be/become the preferred way to search for hyper-parameters for sklearn compatible models. If there is something missing from BayesSearchCV that makes you roll your own then maybe we should fix that?

[jolespin]

Just saw this now. BayesSearchCV is great!

Is n_calls from gp_minimize the same as n_iter from BayesSearchCV? For the surogate, can you give it ensembles?

[betatim]

Is n_calls from gp_minimize the same as n_iter from BayesSearchCV?

Yes, we kept the scikit-learn name in BayesSearchCV for compatibility

For the surogate, can you give it ensembles?

Yes, see for example the parameter for skopt.utils.cook_estimator on how to get a random forest, extra tree or gradient boosted tree ensemble.

[jolespin]

I'm going to close this one out b/c this is exactly what I was envisioning. Thanks guys!