Bazel: Building on Windows

[abergmeier]

The building docs mention to run _MSYS2_ shell, excluding _mingw32_ and _mingw64_. Sadly there is no reason or bug for using anything other than _MSYS2_. Could anyone please elaborate, why the others do not work?

[damienmg]

No reason, wasn't tested. As we stated in our FAQ we are not working on Windows support because we don't have manpower for that at the moment. The support of windows is highly experimental.

[abergmeier]

Created pull request #277 to clarify situation for others, too.

[davidzchen]

FYI one of the major blockers for proper Windows support (that is, without the need to run with elevated permissions) is that Bazel uses symlinks extensively, and while NTFS supports symlinks, creating symlinks requires that the user has SeCreateSymbolicLinkPrivilege. By default, only administrators have this privilege.

This is even further complicated by SeCreateSymbolicLinkPrivilege being subject to UAC.

According to Microsoft, the reason why this restriction exists is that:

Symbolic links (symlinks) can expose security vulnerabilities in applications that aren't designed to handle symbolic links.

I was not able to find any more details or examples of said possible vulnerabilities aside from symlink races when creating symlinks under world-writable directories such as /tmp, though, this is mitigated on POSIX systems using mkstemp(3) and mktemp(1).

Unfortunately, from what I can observe, while symlink support on Windows exists, using the current implementation for most common use cases is pretty much impractical due to this restriction: https://www.google.com/search?q=SeCreateSymbolicLinkPrivilege

[damienmg]

We could do with copying instead of symlinking and that would probably already be a good improvement over standard file system. There's a also the possibility of creating a kernel module to support our own kind of symlink or use the ntfs symlinking with a special service.

Anyway doing a FS abstraction that use copy instead of symlink should work well for a first step.

[davidzchen]

SGTM. While copying instead of symlinking will incur a significant performance hit due to the I/O overhead, it would be a good first step. There is also the issue of Skylark rules that directly invoke ln to create symlinks. Would it be a good idea to have wrapper functions to abstract some of the common file system operations?

[damienmg]

I think file operation abstraction should be provied at the skylark level
so spawn less and less shell scripts.

On Mon, Jul 6, 2015 at 1:02 PM David Z. Chen [email protected]
wrote:

SGTM. While copying instead of symlinking will incur a significant
performance hit due to the I/O overhead, it would be a good first step.
There is also the issue of Skylark rules that directly invoke ln to
create symlinks. Would it be a good idea to have wrapper functions to
abstract some of the common file system operations?

—
Reply to this email directly or view it on GitHub
https://github.com/google/bazel/issues/276#issuecomment-118815425.

[hanwen]

FWIW, I think we could replace symlinks with hardlinks (files) and junctions (directories).

[bjacob]

Just +1 to this issue: I believe that supporting many build platforms is critical to the wide adoption of a build tool like Bazel.

[damienmg]

By the way, we have set-up a goal to have basic windows support by the end of the year so stay tuned!

[kchodorow]

See the roadmap: http://bazel.io/roadmap.html.

[davidzchen]

A gentle ping on this issue. There is strong interest from TensorFlow users to build and use TensorFlow natively on Windows (https://github.com/tensorflow/tensorflow/issues/17).

Perhaps let's give an update on the status of Bazel on Windows?

[ulfjack]

Dmitry has some changes (https://github.com/dslomov/bazel-windows) that allow Bazel to bootstrap. However, he's only working on it part-time; we're planning to make a more concerted push here early next year.

[surban]

@davidzchen While creating symbolic links on Windows requires special permissions, it is possible to create hard links and directory junctions (basically symbolic links to directories) as a standard user. See https://msdn.microsoft.com/en-us/library/windows/desktop/aa365006(v=vs.85).aspx for details. The "mklink" command (in cmd.exe not PowerShell) can be used to quickly test hard links and junctions.

[jakesays-old]

I don't see symlinks on windows being as big an issue as it is being made out to be. The first step should obviously be to get things building on windows with elevated perms. I'm sure pretty much everyone interested in windows support could live with that inconvenience for the short term.

[ulfjack]

Indeed, symlinks is largely solved now using hard links and junctions. But there's still more to do.

[badescuga]

+1 for WIndows support (TensorFlow)

[FabHan]

+1

[rokstrnisa]

+1

[ulfjack]

We have started to upstream Dmitry's changes, as well as done some experiments to check if our approach to symlink handling on Windows is going to work. Experiments look promising and we hope to have bootstrap working for master in the next couple of weeks.

[shysky]

nice

[lberki]

Now that we can _almost_ bootstrap on Windows, I filed a bunch of bugs to track the work that remains to be done in order to call our Windows support usable:

https://github.com/bazelbuild/bazel/labels/Windows

[damienmg]

Thanks Lukacs ! Let's close this one when we can bootstrap on Windows and CI is up and running!