Allennlp: Output config.json in the same order to the input config file

[handsomezebra]

Is your feature request related to a problem? Please describe.
The config.json in the serialization directory looks quite different to the input config file. It is not easy to figure out which input config file was used for one experiment, especially when there are a lot of those experiments.

Describe the solution you'd like
While outputing config.json, can we use the same order of keys as the input config file? This can be accomplished by using an OrderedDict to load the original json.

[DeNeutoy]

Sounds like a good idea! Do you want to send a PR?

[handsomezebra]

I tried but _jsonnet already changed the order before json sees the content. Didn't find a way to do this before _jsonnet supports this.

[matt-gardner]

We could just copy the original config file into the serialization directory, instead of dumping parameters? Why not do that? Hmm, because of overrides... Yeah, this might need a pretty complicated solution to get right.

[joelgrus]

if the goal is to maintain a correspondence between the original config file and the serialized results, wouldn't it be better to just do that directly? (having the keys in the same order still leaves things you have to check). you could just dump some metadata into the archive like "original_config_path", although in some cases you might not want that in there

[matt-gardner]

Hmm, ok, yeah, we could just copy over the original config file with a separate name, if that's the goal. I'm not sure how useful it is, though, and it could be misleading if you ran stuff with --overrides. Should we dump that too?

[handsomezebra]

I think it's not a big problem for now as long as all neccesary information to reproduce the experiments is serialized. In the future it will be fantastic if we have some kind of management console to explore and compare experiments results as below. Thanks for the great tool!

[matt-gardner]

Yeah, this is maybe better handled by beaker (@handsomezebra, that's the tool we use internally to run experiments, which looks like it has similar goals to the tool in your screenshot. We're hoping to release it as open source in the near future). I'd recommend closing this issue.

[HarshTrivedi]

@matt-gardner I am quite curious above having beaker accessible. By any chance the plan is to release it in this summer (next 1-2 months)?

[matt-gardner]

@HarshTrivedi, it's a different team at AI2 that's building beaker, and I can't speak for their plans. I know it's their goal to release the tool, but I don't know timelines.

[HarshTrivedi]

Ok, I see ... Anyways, thanks!