Dvc: Fresh install /.dvc/updater.lock for all commands

[hjuhel-cdpq]

Setup informations

dvc --version
0.71.0

Installed with pip, on a Ubuntu 18.04 version

Issue

Hi,

I am m new to DVC. I have installed DVC on a fresh Ubuntu install. Every command (except init and get) I tried to run raised the following error :

ERROR: unexpected error - [Errno 1] Operation not permitted: '<path_to_repo>/.dvc/tmp/bb2d3250118bf7b0d1042b10868e8e14.lock' -> '<path_to_repo>/.dvc/updater.lock'

With being a path to a working Git repository.

The issue first occured when I was following the "Get Started" section of DVC :

dvc get https://github.com/iterative/dataset-registry/get-started/data.xml -o data/data.xml
dvc add data/data.xml

For now I have tried :

• To remove the updater.loc, but the file does not exist
• To remove the .dvc/tmp/*.lock .

The status is not working neither.

Can you help figuring out what I'm doing wrong please ?

[efiop]

Hi @hjuhel-cdpq !

Could you please install psutil using pip install psutil and then show us dvc version(note that it is not the same as dvc --version) output?

[hjuhel-cdpq]

Hi @efiop

Thanks for your response

My bad, I did not noticed the differences between the two commands.

The output of the command is the following :

DVC version: 0.71.0
Python version: 3.6.9
Platform: Linux-4.15.0-72-generic-x86_64-with-Ubuntu-18.04-bionic
Binary: False
Package: pip
Filesystem type (cache directory): ('fuse.vmhgfs-fuse', 'vmhgfs-fuse')
Filesystem type (workspace): ('fuse.vmhgfs-fuse', 'vmhgfs-fuse')

[efiop]

@hjuhel-cdpq Got it. So looks like you are running on a FUSE fs, which doesn't have hardlink implemented. Related to https://github.com/iterative/dvc/issues/2831 . I suppose it is a virtual machien that you are running and that partition is coming from the host, right?

[hjuhel-cdpq]

Yes you are right. The I'm using a directory mounted on my VM but hosted in the Windows Host.

[efiop]

@hjuhel-cdpq Could you try installing this dev version:

pip uninstall -y dvc
pip install git+https://github.com/efiop/dvc@2831

and then run dvc status please? The dev version is trying to use flock-based locks instead of hardlink-based locks, so if vmhgfs supports flock then everything should start working.

[hjuhel-cdpq]

It's working ! Thanks a lot ! The "add" seems to be working too. Do you plan to permanently use flock locks ?

[efiop]

@hjuhel-cdpq We'll definitely bring back flock by default and either will automatically detect that it is not supported (so far it seems problematic, as it might just hang forever on some nfs versions) or will introduce a config option to use nfs locks. Or, there is one more idea, where we could make the lock dir configurable, so that we could place the locks somewhere outside of the repo. In any case, it is on our TODO list for this sprint, so the permanent solution is coming soon, stay tuned! 🙂 Thanks for the feedback!

[efiop]

@hjuhel-cdpq One more workaround would be to only use fuse mount as an external directory (see dvc cache dir command in our docs), while having the main repo on a normal FS (.e.g ~/ should probably be okay).

[hjuhel-cdpq]

I have start reading about the cache dir in the docs. I am definitely going to try this too.
Thanks again forhaving be so helpfull.