Dvc: type=symlink protected=true does not seem to work

[ematvey]

DVC version: 0.54.1
Python version: 3.7.3
Platform: Linux-4.15.0-55-generic-x86_64-with-debian-buster-sid
Binary: False
Cache: reflink - False, hardlink - True, symlink - True
Filesystem type (cache directory): ('ext4', '/dev/nvme0n1p2')
Filesystem type (workspace): ('ext4', '/dev/nvme0n1p2')

There seems to be a problem when using cache.protected=true togather with cache.type=symlink. During dvc fetch we always have "Operation not permitted" after each link is created. Interestingly, each link fails only once, i.e. after each error it proceeds one link further. DVC seems to be trying to do something with the link or cache entry that is not permitted.

We use shared DVC cache on this machine. All users share primary group. Cache permissions are 2777.

[ematvey]

I'm not 100% sure that this is DVC issue and not something weird in our setup, although this issue reproduces on all of our machines.

[efiop]

@ematvey The issue comes from dvc not being able to chmod files that user doesn't own. We've addressed it in a recent PR, but didn't publish a release with it just yet. I've triggered a new release, it should be ready in an hour or so. So the first step would be to fix permissions of the already existing cache. Please ensure that no one is using the cache while you are doing these steps.

find /path/to/cache/dir -type f -exec sudo chmod 0444 {} \;
find /path/to/cache/dir -type d -exec sudo chmod 0775 {} \;

Now enable group sharing mode in _all_ dvc project instances that are using this cache with

dvc config cache.shared group

After this, pull and other operations should work correctly.

[ematvey]

This fix seems to work (tested with dvc from master), closing the issue.

[efiop]

@ematvey Btw 0.55.0 is out, so please feel free to upgrade :slightly_smiling_face:

[ematvey]

Thx, we've already installed it from sources. Not on pypi yet.

[efiop]

@ematvey oh, indeed, there was a malfunction in our deployment system and pypi package wasn't deployed. Fixing it right now. Thanks!