Hydra: [Bug] Compose API does not parse Hydra configs

[msalvaris]

🐛 Bug

When using the experimental Compose API the configs for hydra don't seem to be parsed. This manifests in my particular instance as the logging not being correct.

Expected Behavior

That the hydra configs are parsed and it behaves the same as calling function decorated with hydra.main

System information

• Hydra Version : 0.11.3
• Python version : 3.6
• Virtual environment type and version : Anaconda
• Operating system : Ubuntu 16.04

[omry]

This is not really a bug, the compose API is a much more lightweight version that does not do many things @hydra.main() is doing.
among others:

• No CLI integration
• No working directory management
• No multirun

Can you explain your use case?
Why are you using compose and not @hydra.main()?

[msalvaris]

I thought as much so kind of vacillated between bug and feature
I am trying to use an AWS service that passes in parameters as --param=value. Hydra takes in parameters as follows param=value. Wanted to see if I could create an alternate entrypoint that would allow my app to parse params starting with --.

[omry]

I see.
You can look at the code the hydra.experimental.compose() is calling to figure out how to configure the logging yourself as an unsupported solution (it may break in future versions as you will be using an internal API).

Why do you need an entry point that parses -- params?

[msalvaris]

This is how the AWS service passes arguments to your script. It takes a dictionary of variables and passes them to your script with -- appended at the front. It would be nice if it didn't do that but just wanted to see if the workaround would be easy.

Yea, I will give the custom solution a go. Thanks!

BTW really loving Hydra, have implemented pipelines using it in both TensorFlow and PyTorch. Used Yacs before.

[omry]

awesome.
Maybe file an issue to Amazon to to allow passing parameters without the --.
Can you give a link to the "AWS service" you are referring to?

BTW:
I hope that Hydra will get some kind of AWS launching capabilities in the future.

[msalvaris]

Yea, will definitely do that
The service is Sagemaker https://sagemaker.readthedocs.io/en/stable/estimators.html

I hope that Hydra will get some kind of AWS launching capabilities in the future.

That would be awesome! :rocket:

[omry]

Closing for now as this is by design.