Dvc: long running dvc add does not print a message to use symlinks, hardlinks

Created on 17 Sep 2019  路  3Comments  路  Source: iterative/dvc

Comes from a discussion on ODS. User was trying to add 4TB data.

Check that it's actually happening. Or may be message is just not visible enough?

Check other operations that are affected by cache type (checkout, pull).

Write tests to make sure we don't break it in the future again.

enhancement p1-important research ui

Most helpful comment

Should we raise it to p1? Since we onto UI somewhat nowdays)

All 3 comments

Ok, so it seems that the problem with "Consider other link types" warning is that it shows up when a single linking operation takes more than 10 seconds. Clearly, that applies only to copying huge singular files. If the user adds a directory, the limit might never be reached, and the message might not be displayed. We should probably consider how to apply slow_link_guard to directory-related operations. The first thing that comes to my mind, would be to consider slow_link_guard as a static object that would aggregate time spent on linking.

Now that I wrote it down, that does not sound elegant.

Should we raise it to p1? Since we onto UI somewhat nowdays)

Was this page helpful?
0 / 5 - 0 ratings

Related issues

nik123 picture nik123  路  3Comments

shcheklein picture shcheklein  路  3Comments

tc-ying picture tc-ying  路  3Comments

mfrata picture mfrata  路  3Comments

dmpetrov picture dmpetrov  路  3Comments