Bazel: Windows: support long paths when calling WinAPI functions

[laszlocsomor]

Description of the problem / feature request / question:

We're hitting path length limitations because we use the ASCII version of Win32 functions and paths are limited to 260 characters (MAX_PATH).

See for example https://github.com/bazelbuild/bazel/issues/2178#issuecomment-264846530.

We have to use the widechar versions and prefix all long paths with \\?\.
Caveats: such prefixed paths may not use / as the path separator, only \. Using / only works with the ASCII versions.

If possible, provide a minimal example to reproduce the problem:

$ git checkout c9e363c674e4a9cd2422e80da2636f6620798fcb
$ bazel test //src/test/shell/bazel:bazel_windows_example_test

Environment info

• Operating System: Windows (every version)

• If bazel info release returns "development version" or "(@non-git)", please tell us what source tree you compiled Bazel from; git commit hash is appreciated (git rev-parse HEAD):

$ git rev-parse HEAD
c9e363c674e4a9cd2422e80da2636f6620798fcb

Have you found anything relevant by searching the web? (e.g. GitHub issues, email threads in the [email protected] archive)

Naupe.

Anything else, information or logs or outputs that would be helpful?

(If they are large, please upload as attachment or provide link).

[laszlocsomor]

Bumping to P1 because this makes our CI flaky / failing.

[petemounce]

Anything I can do to assist with this? It's one of the things that blocks our Windows adoption.

[laszlocsomor]

Thanks, I think we're close to fixing this. I'm about to update the CreateFileA calls to CreateFileW in blaze_util_windows, then update the CreateProcessA ones to W, and that should do it.

[laszlocsomor]

TIL: CreateProcessW doesn't support long paths, not even with "\\?\".

(..a moment of silence..)

Luckily, there's GetShortPathName so we can shorten those long paths and CreateProcess can spawn a process with them.

[laszlocsomor]

Another poor soul who ran into this, and tried everything I tried and some more:
https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/944e2977-ad09-4b10-818b-ba383a3f8db0/createprocess-with-extended-length-path-fails-errorinsufficientbuffer?forum=windowsgeneraldevelopmentissues

[petemounce]

Happy New Year...?

[laszlocsomor]

I have a change that fixes about 5 long path related bugs. Now I can build bazel with a long output_user_root. I'll break it down and start sending out.

[dslomov]

Is there still work to do here?

[laszlocsomor]

Not that I'm aware of. It's my great pleasure to hit "Close and comment".

[petemounce]

We're going to eagerly try this out this week as it's one of the things that blocks our use of Bazel on Windows. Thanks and we'll keep you posted!

[laszlocsomor]

We're also going to start dogfooding the MSYS-less version with 0.5.0, stay tuned!