Dvc: get ouput files from different branches (with some code)

[mastaer]

Why Should I want that?:

I want this to simply get all files from multiple experiments (one branch == one experiment (or one sub-experiment)).

Scenario 1:

I might want to use Adam Boost from several models from different branches to get a model that worked with all the sub models. So I have to get all the models first. And this you could do with this script.

Scenario 2:

I did several experiments with Tensorflow and saved the results like the step acc in Tensorboard. This is a big file and I don't want to use it all the time. So I think it's a cached output file. But sometimes I want to do more than just compare the metrics. Then I could run this script to get all Tensorboards of interest and run Tensorboard to evaluate this in detail.

Should be this a part of DVC?

I would really love this feature, but I also love the simplicity of dvc. If more and more commands are coming, than DVC get complexer and harder to understand for new users. But I still think that this could be a part of DVC, because it can be seen as an extension of the dvc metrics show . With this command we can already take a look at all the branches. This script would extend this so that the files can be linked in a tmp file and analyzed with an external tool, as was the case in scenario 2.

Script

You can find a script that already does this, but is currently not part of DVC here: see below

If the code does not work for you, just let me know.

Chat channel: q-and-a ; date: 05/16/2019

There are already a discussion in the chat (I just coppied, my message here). I don't know how to create a link to discordapp.

[mastaer]

The script changed a bit and I plan to add a parameter that you don't need. The new path to the script is:
https://github.com/deep-projects/dvc-cc/blob/master/dvc-cc/dvc_cc/output_to_tmp/main.py

[mastaer]

for this script the issue: https://github.com/iterative/dvc/issues/2146 , needs to be fixed

[efiop]

btw @mastaer , we have a new feature called dvc get, that is able to get you (download) any file from any repo. E.g. you could do something like dvc get /path/to/myrepo path/to/file/within/repo --rev mybranch.

[mastaer]

Hey guys,
dvc get was exactly what I needed. Currently, I think that the idea of DVC is not to work with multiple git branches. So if you want, you can close this feature request.
you are great

[efiop]

@mastaer Thanks for the feedback! Closing for now.