cabal dies when creating pkg registration file (Windows)

[coot]

Describe the bug
Building on Windows (using cabal-3.0) I am getting the following error:

cabal v2-build Win32-network
...
Creating package registration file:
\\vboxsrv\iohk\ouroboros-network\dist-newstyle\tmp\package-registration--6792\pkgConf
\\vboxsrv\iohk\ouroboros-network\dist-newstyle\tmp\package-registration--6792\: openBinaryTempFileWithDefaultPermissions: does not exist (No such file or directory)
CallStack (from HasCallStack):
  die', called at .\\Distribution\\Client\\ProjectOrchestration.hs:1035:55 in main:Distribution.Client.ProjectOrchestration
cabal.exe: Failed to build Win32-network-0.1.0.0-inplace. The failure occurred
during the final install step.

Expected behavior
A clear and concise description of what you expected to happen.

System information

• Operating system: Windows 10
• cabal-3.0, ghc-8.6.5

Additional context
Add any other context about the problem here.

[coot]

A workaround this is to use a builddir on a regular drive: cabal v2-build --builddir /c/dist ... (in git-bash).

[coot]

The reason for this problem lies in the usage of System.Directory.canonicalizePath. In my setup dist-newstyle is on a network drive iohk (\\VBOXSvr) (Z:). For a file which relative path is in that directory canonicalizePath returns:

>>> canonicalizePath "some.file"
"\\\\VBoxSvr\\iohk\\some.file"

However the makeAbsolute returns the path:

>>> makeAbsolute "some.file"
"Z:\\some.file"

Assuming that the file Z:\iohk\some.file (aka \\VBoxSvr\iohk\some.file) exists

>>> makeAbsolute "some.file" >>= doesFileExist
True
>>> canonicalisePath "some.file" >>= doesFileExist
True

>>> canonicalisePath "some.file" >>= \path -> openFile path ReadMode
*** Exception: \\VBoxSvr\iohk\test.file: openFile: does not exist (No such file or directory)

while we can still open with the absolute path (Z:\iohk\some.file):

>>> makeAbsolute "some.file" >>= \path -> openFile path ReadMode

Does cabal relies on canonicity of paths, or could we switch to makeAbsolute?

[phadej]

canonicalisePath "some.file" >>= doesFileExist
True

but opening fails?

[coot]

but opening fails?

Yes. If you want to reproduce: start a VirtualBox and try to compile something in a shared folder.

[phadej]

@Mistuke is there a short explanation why openFile and doesFileExist behave differently?

[Mistuke]

@phadej openFile goes through GHC and we seem to have a bug in the remapping code for network shares https://gitlab.haskell.org/ghc/ghc/blob/master/utils/fs/fs.c#L96 can you file an upstream ticket and assign it to me? I'll fix it tonight

[phadej]

@Mistuke, done: https://gitlab.haskell.org/ghc/ghc/issues/17788

[Mistuke]

@phadej Thanks, started a bootstrap and regression test for this.

[Mistuke]

To clarify, this is a regression in GHC 8.6.5 where we started remapping paths to use full namespaced paths in GHC in order to get around MAX_PATH. For a Win32 path such as C:\foo we rewrite it to \\?\C:\foo. For a network share \\foo\bar this needs to become \\?\UNC\foo\bar but we were accidentally making it \\?\UNC\\foo\bar notice the double \\. That's an invalid path.

doesFileExist never touches the GHC I/O manager, though it does internally do the same rewriting. It doesn't suffer from the same bug so it correctly returned true.

[coot]

@Mistuke do you think the fix will be included in ghc-8.6.6?

[Mistuke]

I don't think there's going to be a another 8.6 release

[angerman]

I don't think there's going to be a another 8.6 release

But the .6 releases are always the good ones.