Ax: Passing extra arguments to the evaluation_function

[arvieFrydenlund]

I am trying to do a hyper-parameter search over a subset of the overall model hyperparameters. This subset has been defined for the search space, however, my evaluation_function needs access to the rest in order to pass them to the model's trainer. I feel like this might conflict with Ax's design philosophy but I have a considerable number of these and its annoying to define them both through argparse and then again through ax.

1) Is there a way of passing these to the evaluation_function function without creating a parameter for them in the search space i.e. having them as part of the parameterization?

2) If not is there a helper function which converts and argparse to ax parameters? or something similar.

Thanks.

[eytan]

Would “fixed parameters” accomplish what you are looking for? https://ax.dev/docs/core.html

Sent from my iPhone

On May 5, 2019, at 11:50 AM, Arvie Frydenlund [e-mail] wrote:

I am trying to do a hyper-parameter search over a subset of the overall model hyperparameters. This subset has been defined for the search space, however, my evaluation_function needs access to the rest in order to pass them to the model's trainer. I feel like this might conflict with Ax's design philosophy but I have a considerable number of these and its annoying to define them both trough argparse and then again though ax.

Is there a way of passing these to the evaluation_function function without creating a parameter for them in the search space i.e. having them as part of the parameterization?

If not is there a helper function which converts and argparse to ax parameters? or something similar.

Thanks.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[arvieFrydenlund]

That kind of helps but not fully.

I can partially use that to convert an argparse object to a list of fixed parameters, however, there are some types that can't be converted, for example, there isn't a ParameterType for list (which is not being used as a range).

[amurthy1]

If these other parameters are fixed for all trials, can they be encoded directly in the evaluation_function itself rather than passed as arguments? You could even write a wrapper function which takes the fixed params as inputs and outputs an evaluation_function with them encoded inside.

If by argparse you mean an easy to use CLI, that is something we've discussed supporting and would be interested to know how you're using it with CLI.

[arvieFrydenlund]

Thanks, the wrapper function worked.

And yes this was for CLI. I'm trying to do Bayesian optimization on a pre-existing library which uses CLI to define the model and I was trying to integrate ax with minimal changes to the existing code.

[amurthy1]

Ah so not invoking Ax via CLI. Glad this solution worked for you.