Hyperopt: [question] use parameter values to define search space for subsequent parameters

[flokde]

Dear hyperopt team,

I am currently using hyperopt to optimise over a rather awkward search space to find an optimal sklearn pipeline for imbalanced classification. I really enjoy the flexibility when defining the search space. Is it possible to use a chosen parameter value to define the search space for other parameters? Say my first parameter is chosen at random between zero and one. I want my second parameter to be chosen at random between zero and the value chosen for the first parameter. For example:

I have random downsampling in my pipeline as the first step, which uses the desired ratio majority/minority class after the sampling is done as a hyperparameter. I have then another resampling preprocessing step and would like to use the value chosen for this first hyperparameter (the ratio) to set a bound for the next hyperparameter.

I couldn't definitively tell from the documentation and was wondering if this is possible.
Thank you very much in advance!

[bjkomer]

There isn't a nice way to do this directly that I am aware of, but a similar result can be achieved by redefining the second parameter as a proportion of the first. In this manner, you can calculate the actual value of the second parameter in the objective function, and no explicit dependency needs to be specified.

For example, something like this:

space = {
    'p1': hp.uniform('p1', 0.0, 1.0),
    'p2': hp.uniform('p2', 0.0, 1.0)
}

def objective(params):
    p1 = params['p1']
    p2 = params['p1'] * params['p2']
    # ...

[flokde]

That is actually quite elegant - I just didn't think of doing it that way. Thanks so much for the quick reply and support!