Hydra: [Feature Request] support use case of composing the same config node in multiple places in the config tree

[YannDubs]

🚀 Feature Request

Be able to copy all the elements of one config in an other.

Motivation

In my code I have 2 modules which are trained, meaning that I have 2 optimizers. I would like to modify the parameters of one optimizer, but don't want to copy have all default optimizer values written twice. I.e would like to do that python main.py module1.optimizer=adam module1.optimizer.lr=0.01 module2.optimizer.lr=0.01 but not have to define a optimize twice. Something like that:

module1:
  ...
  optimizer: ${optimizer.adam}

module2:
  ...
  optimizer: ${optimizer.adam}

which would use the adam optimizer as default for module 1 and 2 without having to dupicate it.

[omry]

The current model in Hydra does not support this kind of thing well.
One thing to try is something with interpolating config nodes, which I mentioned in your other issue:

defaults:
  optimizer: adam

module1:
  ...
  optimizer: ${optimizer}

module2:
  ...
  optimizer: ${optimizer}

I will keep this open as a reminder to improve support for this use case.

[YannDubs]

In the other thread you say "You can interpolate a node, not just a value" which is what I would like (if I understand what you mean). Can you give an example of how to do that ?

Because in the code above, ${optimizer} is simply the string adam not a dictionary/node right?

[omry]

optimizer is not the string adam, it's the node for adam optimizer that is in optimizer/adam.yaml.

[omry]

I am actually not 100% sure how this would work if you start overriding things, but give it a try and let me know.

[YannDubs]

I am actually not 100% sure how this would work if you start overriding things, but give it a try and let me know.

I just tried : it overrides the main optimizer (i.e. it does not copy the node), so what I was saying is not possible python main.py module1.optimizer.lr=0.1 module2.optimizer.lr=0.01 sets the lr to 0.01 (the last value used) for the main optimizer => also for both modules

[YannDubs]

I don't see any usecases for interpolating the node if it's not copied, right ? if so maybe it would be worth copying instead ?

[omry]

One use case is to make the values of the node accessible in multiple places, much like the use case value interpolation.

Something that can be implemented here is a copy function, but this may be harder than it seems:

module1:
  ...
  optimizer: ${copy:optimizer}

I will keep this open to try this later.
If you would like to try look at custom interpolations in OmegaConf.
As I said, although returning a copy may sound trivial it may actually be quiet tricky and have some weird side effects.
It's easy enough to try.
Don't use the function name copy though to avoid conflict with what I will try to implement later.

maybe copy_node?

[omry]

This should probably be implemented in OmegaConf. there is a good chance that it's not easily possible to do externally at the moment.

[omry]

until this is implemented, a workaround can be implemented as a symlink at the file system level:
optimizer1/{adam,nexterov,sgd>.yaml
optimizer2 -> optimizer1

[omry]

another idea I had which might be easier to implement is to use the defaults to do this by adding a few optional fields (source, dest, and group).

for example:

defaults:
   - optimizer: adam
     source: optimizer
     dest: optimizer1
   - optimizer: adam
     source: optimizer
     dest: optimizer2

To make it possible to override from the command line, maybe add another optional field for the group name:

defaults:
   - optimizer1: adam  # identifier optimizer1, with value adam
     group: optimizer  # group optimizer, this means optimizer/adam.yaml will be used
     source: optimizer # this is the source path in the config
     dest: optimizer1  # this is the destination path. when included, the optimizer node will be added ad optimizer1
   - optimizer2: adam
     group: optimizer
     source: optimizer
     dest: optimizer2

This is not as elegant as previous suggestions but it gives quiet a bit of flexibility.
Thoughts?

[omry]

Another idea to play with:

group@path=name

or in defaults:

defaults:
  - db@src_db: mysql
  - db@dst_db: mysql

I kind of like this one.

[isvoboda]

I am in a similar situation composing my configurations. My use-case include predefined configuration of an experiment based on the suggestion from https://github.com/facebookresearch/hydra/issues/386#issuecomment-589586845

I do not know how to redefine the group model in defaults:

• conf/config.yaml
  ~yaml
  defaults:
  model: UNetPix
  ~

• conf/model/UNetPix.yaml

~yaml
model:
class: models.UNet.UNetPix
params:
kwarg1: ...
kwarg2: ...
~

• conf/model/UNetTC.yaml

~yaml
model:
class: models.UNet.UNetTC
params:
kwarg1: ...
kwarg2: ...
~

I would like to add configuration of an experiment deploying the UNetTC instead of UNetPix in such a way I can just call python train.py experiment=UNetTC.

• conf/experiment/UNetTC.yaml

~yaml
model: UNetTC # Here I would like to change UNetPix by UNetTC group defined in model/UNetTC.yaml
... # other experiment related configurations
~

Is this currently possible?

[omry]

Your question does not seem related, please ask it in the chat or open a new issue.

[isvoboda]

I would say it is a particular case related to the "composing the same config node in multiple places". I have opened a new issue https://github.com/facebookresearch/hydra/issues/582#issue-615246165