Dvc: Investigate TestCmdCacheDir failing on MacOs and non-realpath repo.

[pared]

In https://github.com/iterative/dvc/pull/4673, on MacOs we need to expand the temporary directory to its real path in order to make a test for relative cache dir setup pass. I tried to investigate why it fails without os.path.realpath and came up with following test:

def test_cmd_cache_dir_relative_path(make_tmp_dir):
    d = make_tmp_dir("temp")
    pv = d / "private" / "var"
    os.makedirs(pv)

    l = d / "var"
    System.symlink(pv, l)

    cache_path = l / "cache"

    repo_path = l / "repo"
    os.makedirs(repo_path)
    with repo_path.chdir():
        repo = Repo.init(no_scm=True)

        dname = relpath(cache_path)
        assert main(["cache", "dir", dname]) == 0

        rel = os.path.join("..", dname)
        config = configobj.ConfigObj(repo.config.files["repo"])
        assert config["cache"]["dir"] == rel.replace("\\", "/")

        repo_path.gen("foo", "foo content")
        assert main(["add", "foo"]) == 0

The test seems to pass both on Linux and MacOs.

Investigate why TestCmdCacheDir fails if we don't wrap self.mkdtemp in os.path.realpath in case of MacOs.

MacOs version of failing runner:
Mac OS X - 10.15.7
DVC version: 1.8.4

[pmrowla]

It looks like there's an issue with os.mkdir has issues when you mix relative paths (with ..) and symlinks on mac? The original test fails when we use os.makedirs() to generate cache directories

Python 3.8.6 (default, Oct  8 2020, 14:06:32)
[Clang 12.0.0 (clang-1200.0.32.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tempfile
>>> import os
>>> temp = tempfile.mkdtemp()
>>> temp
'/var/folders/s_/j8cg24s945dcvlh77c677f080000gn/T/tmpn58qq3u5'
>>> rel = os.path.relpath(temp)
>>> rel
'../../../var/folders/s_/j8cg24s945dcvlh77c677f080000gn/T/tmpn58qq3u5'
>>> os.mkdir(os.path.join(temp, "test1"))
>>> os.mkdir(os.path.join(rel, "test2"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '../../../var/folders/s_/j8cg24s945dcvlh77c677f080000gn/T/tmpn58qq3u5/test2'
>>> real = os.path.realpath(temp)
>>> real
'/private/var/folders/s_/j8cg24s945dcvlh77c677f080000gn/T/tmpn58qq3u5'
>>> os.mkdir(os.path.join(real, "test3"))
>>> real_rel = os.path.relpath(real)
>>> os.mkdir(os.path.join(real_rel, "test4"))
>>> real_rel
'../../../private/var/folders/s_/j8cg24s945dcvlh77c677f080000gn/T/tmpn58qq3u5'

[pmrowla]

I played around with it a bit more and it looks like the issue is specific to the /private/... symlink setup (and relpaths with links to other paths work as expected). Seems like the solution is to just do the realpath(mkdtemp(...)) wrap like you've already implemented in the cache directory test.

[pared]

@pmrowla thanks for researching the subject, I have to admit that doing that without mac is troublesome. In that case I guess we can close that one.