Hydra: Recursive defaults support

[omry]

Hydra does not currently support a recursive defaults list. the defaults are only specified at the primary config file.
There is some special support for Hydra itself, but this can be ignored for the purpose of this issue.

Recursive defaults list are tricky, it's important to ensure intuitive and consistent order of merging.

Proposed logic

if the defaults in the primary file are in that order:
main.yaml

defaults:
  - aa: x1
  - bb: x2

The relative order should be preserved. it is however possible for other files to be injected in between aa and bb, for example if aa/x1 had it's own defaults:

aa/x1.yaml

defaults:
  - cc: x3
  - dd: x4

the resulting defaults list might looks like:

  - aa: x1
  - cc: x3  # from aa/x1 
  - dd: x4  # from aa/x1 
  - bb: x2

Let's look at a more complicated case:

main.yaml

defaults:
  - aa: x1
  - bb: x2
  - cc: must_win

aa/x1.yaml

defaults:
  - cc: x3
  - dd: x4

In this case, the result should be:
the resulting intermediate list will looks like:

  - aa: x1
  - cc: x3
  - dd: x4
  - bb: x2
  - cc: must_win

but the final result will be:

  - aa: x1
  - cc: must_win
  - dd: x4
  - bb: x2

Note how cc retains it's order in the list, but got the last value.

The rule here is:

1. Composition order determined by the first time a configuration group is introduced (the db in db=mysql).
2. Composition choice (the mysql in db=mysql) is determined by the last mention of the group).

[Queuecumber]

Would like to see this, I have a bunch of options for config groups, when I find the ones that work well using command line overrides I'd like to dump them into a config file so I don't have to worry about the long command line again.

[omry]

The plan is to implement this for 1.1.

[ppwwyyxx]

How does recursive defaults work with package overrides? e.g. something like this:

# model/maskrcnn.yaml:
# @package _group_
a: 1
b: 2

# model/maskrcnn2.yaml:
# @package _group_
b: 3
c: 4

defaults:
  - model/maskrcnn   # ??
  - _self_

# main.yaml:
defaults:
  - [email protected]: maskrcnn2

[omry]

The key for overriding becomes group@package is you have a package override.
This should work in a similar way to how package overrides are handled from the command line.

[ppwwyyxx]

I can't seem to make the above example work with the commit that's just added. I ended up with

{'model': {'a': 1, 'b': 2, 'submodule': {'b': 3, 'c': 4}}}

[omry]

Looks like you are adding the model in two places.
If you want to rename the existing model, you need to use the rename syntax.

This is documented here:
https://github.com/facebookresearch/hydra/blob/master/website/docs/advanced/defaults_list.md

(Unfortunately this is not in the website yet due to some issue we are looking into).

If you think you have found a bug or have a question about the recursive defaults open a new issue.

[omry]

You can look at the resulting defaults list (and some other info with --info).

[omry]

I am not ruling out a bug though, play with it and if you think there is a bug open a new issue with a minimal example, along with what config you are trying to compose (which is missing here).

[mwillwork]

@omry is there an ETA for this feature and for Hydra 1.1?

[omry]

This feature (or more accurately the one from #1170 a complete rewrite that is not merged yet) is more or less ready (mostly only documentation and some followups are missing).

There are a number of other things which are also planned for Hydra 1.1.
You can follow the milestone for 1.1 here.

I am guestimating that initial release candidate will be ready sometimes during H1 2021.

Keep in mind that Hydra 1.1 will take even longer to hit fbcode due to the monorepo nature of it.