dvc version: does not follow symlinks

Created on 9 Dec 2020  路  6Comments  路  Source: iterative/dvc

Bug Report

Description

This is the dvc version output, where it says the cache directory is nfs4 on storage:/home and cache type is symlink.

DVC version: 1.10.2 (pip)
---------------------------------
Platform: Python 3.8.3 on Linux-5.4.0-54-generic-x86_64-with-glibc2.10
Supports: All remotes
Cache types: symlink
Cache directory: nfs4 on storage:/home
Caches: local
Remotes: s3
Workspace directory: nfs4 on storage:/home
Repo: dvc, git

However, I do have a ~/.config/dvc/config file that overrides this:

[core]
    experiments = true
[cache]
    type = "reflink,symlink,copy"
    protected = true
    dir = /home/jc/ssd_cache/dvc_cache
[feature]
    parametrization = true

And the actual cache dir is /home/jc/ssd_cache/dvc_cache as I've specified instead of nfs4 on storage:/home

awaiting response question

All 6 comments

@johnnychen94 storage:/home is not a path to the cache directory, but the partition that it belongs to. Makes it easy for us to tell if your cache and workspace are on the same partition and so on the same filesystem. From docs https://dvc.org/doc/command-reference/version

Filesystem type (e.g. ext4, FAT, etc.) and drive on which the cache directory is mounted

@johnnychen94 To show current cache dir, run dvc cache dir

Oh, I see the issue here.

/home/jc/ssd_cache is a symlink to /mnt/cache/home/jc, where /mnt/cache/ is a local ssd raid with ext4 format.

Perhaps dvc version needs to follow the symlinks?


By modifying it to /mnt/cache/home/jc/dvc_cache, the results show up normally

DVC version: 1.10.2 (pip)
---------------------------------
Platform: Python 3.8.3 on Linux-5.4.0-54-generic-x86_64-with-glibc2.10
Supports: All remotes
Cache types: symlink
Cache directory: ext4 on /dev/sda3
Caches: local
Remotes: s3
Workspace directory: nfs4 on storage:/home
Repo: dvc, git

@johnnychen94 Ah, yeah, makes sense! I think we just need to change https://github.com/iterative/dvc/blob/a7d665157cd838a9406e30c249065111d93a9e4a/dvc/info.py#L145
to path = pathlib.Path(path).resolve() and that should make it work as expected.

Yes, just verified that this change works.

@johnnychen94 Mind sending a PR? :slightly_smiling_face: :pray:

Was this page helpful?
0 / 5 - 0 ratings