Dvc.org: Document pushing from remote cache will have no effect

Created on 14 Oct 2020  路  5Comments  路  Source: iterative/dvc.org

In https://github.com/iterative/dvc/issues/4686 user got confused, when push for repository with external cache had no effect and did not copy the data to the remote.

We already mention that we need an external cache for external outputs. We should probably include the information that this cache is taking care of versioning those outputs, and we assume that the user will not want to move external outputs from the already configured cache.

doc-content enhancement good first issue hacktoberfest priority-p2

All 5 comments

We need an external cache for external dependencies? This cache is taking care of versioning those dependencies? We mean outputs, right? 馃檪

assume that the user will not want to move external outputs from the already configured cache.

Indeed thit is not expressed in docs. I'm not sure I knew this before actually... So you can't push external outputs? Will --store work to change this behavior? (See iterative/dvc/pull/4581)

Thanks @pared

@jorgeorpinel Yeah, I meant outputs, sorry for the confusion.
As to iterative/dvc/pull/4581 : I don't think it is something related to current issue: OP's problem was inability of pushing external outputs from external cache to remote - while iterative/dvc/pull/4581 is about checkout and import.

@pared Just to confirm, so there's no way to push external outputs, right?

I think iterative/dvc/pull/4581 is definitely related: it introduces a store: true option to outs in dvc.yaml that means that they can be pushed to remote storage. The value would be true by default for all outputs except if they are imports according to that issue. But you are bringing up another case where it seems the default value would always be false...

@jorgeorpinel

Just to confirm, so there's no way to push external outputs, right?

Yes, that is true.

So as to iterative/dvc#4581: This is about data that is stored in local cache. Essentialy this change lets you decide whether you want to synchronize your import -ed data with your remote. The problem from iterative/dvc#4686 is about external output, that has been saved in external cache. This is a special situation (pushing data from external cache to remote), which, as mentioned in discussion is not being supported by DVC. So even after --store, this scenario will not be supported.

So even after --store, this scenario will not be supported.

Great, thanks. I just wanted to know the answer, so we don't give users false expectations when we introduce store: false. (To a user pushing is pushing, whether it's from a local or external cache.)

Was this page helpful?
0 / 5 - 0 ratings