Dvc: import: allow downloading regular files/dirs tracked by Git?

Created on 28 Nov 2019  路  4Comments  路  Source: iterative/dvc

Continuation of #2515.

Per https://github.com/iterative/dvc/pull/2837#issuecomment-559559857, this will still be pending after finishing that ticket.

enhancement feature request p1-important

Most helpful comment

Hi, I would like to work on this, as discussed with Ivan.

Below are some notes to self, please disregard.

import doesn't use get internally: https://github.com/iterative/dvc/pull/2837#issuecomment-559559857

Similar functionality for get: https://github.com/iterative/dvc/pull/2837/files

From https://github.com/iterative/dvc/issues/2515#issuecomment-557593944:

Import doesn't support regular files IIRC. Cached directories are supported, so it would be great to support non-cached ones too as a part of this ticket

Recursive option doesn't exist for any of the commands, so it is not part of this ticket (though I'm not even sure what jorgeorpinel meant by recursive, maybe he could clarify)

From https://github.com/iterative/dvc/issues/2515#issuecomment-557599004

It's really about files hosted in the Git repo vs. files referenced in the DVC-files hosted in the Git repo, right?

Docs:
https://dvc.org/doc/command-reference/get
https://dvc.org/doc/command-reference/import
https://dvc.org/doc/user-guide/dvc-file-format

All 4 comments

Hi, I would like to work on this, as discussed with Ivan.

Below are some notes to self, please disregard.

import doesn't use get internally: https://github.com/iterative/dvc/pull/2837#issuecomment-559559857

Similar functionality for get: https://github.com/iterative/dvc/pull/2837/files

From https://github.com/iterative/dvc/issues/2515#issuecomment-557593944:

Import doesn't support regular files IIRC. Cached directories are supported, so it would be great to support non-cached ones too as a part of this ticket

Recursive option doesn't exist for any of the commands, so it is not part of this ticket (though I'm not even sure what jorgeorpinel meant by recursive, maybe he could clarify)

From https://github.com/iterative/dvc/issues/2515#issuecomment-557599004

It's really about files hosted in the Git repo vs. files referenced in the DVC-files hosted in the Git repo, right?

Docs:
https://dvc.org/doc/command-reference/get
https://dvc.org/doc/command-reference/import
https://dvc.org/doc/user-guide/dvc-file-format

Thanks a lot Wojciech! Your internal notes are a really great summary for this issue BTW. Definitely worth having on record 馃檪

import doesn't use get internally

Should it? Cc @shcheklein @mroutis

Should it?

@jorgeorpinel Not necessarily, they have pretty different implementations right now and I doubt PR for this issue will have to change that. But I guess we'll soon see 馃檪

I have a patch at https://github.com/iterative/dvc/pull/2889 but it depends on https://github.com/iterative/dvc/pull/2837, so I have marked it as Draft/WiP for now. I don't see how to better manage dependencies on GitHub but I might have missed it.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

ghost picture ghost  路  3Comments

prihoda picture prihoda  路  3Comments

GildedHonour picture GildedHonour  路  3Comments

anotherbugmaster picture anotherbugmaster  路  3Comments

ghost picture ghost  路  3Comments