Ax: Hierarchical search spaces

[yonatanMedan]

Is there a way to define a hierarchy of parameters?
for example a parameter that chooses architecture, and each architecture has its own parameters.

example (pseudo code):

architecture = choise(["NeuralNetwork","xgdboost"])

if architecture=="NeuralNetwork":
     n_layers = choise(range(1,10,1))
     #more architecture releted params here.

else if  architecture=="xgdboost":
    max_depth =  choise(range(1,5,1))
     #more architecture releted params here.

[ldworkin]

Hi @yonatanMedan! Great question. We don't currently support this, but it's on our roadmap to support in the next few months. I'll let you know when it's ready!

[riyadparvez]

Yes, this would be a great addition! I have a similar usecase - after hyperparameter optimization choose the right threshold for classification.

[Tandon-A]

This enhancement would be super helpful in my use case where I want to experiments with different learning rate schedulers, where the parameters used by the schedulers are different.

[LyzhinIvan]

Hi! Are there some estimates when this functionality will be available?

[ldworkin]

Hi @LyzhinIvan ! Unfortunately, probably not in the immediate short-term. This has been deprioritized in favor of other efforts. However it's certainly still on our roadmap! cc @2timesjay

[lena-kashtelyan]

We will now be tracking wishlist items / feature requests in a master issue for improved visibility: #566. Of course please feel free to still open new feature requests issues; we'll take care of thinking them through and adding them to the master issue.