Bazel: git_repository does not always work on Windows

[jchv]

Description of the problem / feature request:

git_repository seems to sometimes have issues on Windows. Specifically, this instantiation from bazel-integration-testing is giving me issues (please disregard the fact that bazel-integration-testing doesn't yet support Windows:)

git_repository(
    name = "bazel_skylib",
    commit = "ff23a62c57d2912c3073a69c12f42c3d6e58a957",
    remote = "https://github.com/bazelbuild/bazel-skylib",
)

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

PS > git clone [email protected]:bazelbuild/bazel-integration-testing.git
PS > cd bazel-integration-testing
PS > bazel test ...

ERROR: error loading package '': Encountered error while reading extension file 'lib.bzl': no such package '@bazel_skylib//': Traceback (most recent call last):
        File "C:/bazel/6dd6b3zb/external/bazel_tools/tools/build_defs/repo/git.bzl", line 166
                _clone_or_update(ctx)
        File "C:/bazel/6dd6b3zb/external/bazel_tools/tools/build_defs/repo/git.bzl", line 72, in _clone_or_update
                fail(("error cloning %s:\n%s" % (ctx....)))
error cloning bazel_skylib:
+ cd C:/bazel/6dd6b3zb/external
+ rm -rf C:/bazel/6dd6b3zb/external/bazel_skylib C:/bazel/6dd6b3zb/external/bazel_skylib
/usr/bin/bash: line 4: rm: command not found

What operating system are you running Bazel on?

Windows 10 Pro 1803, MSys2 with Bash 4.4.19.

What's the output of bazel info release?

release 0.17.2

Have you found anything relevant by searching the web?

Searched, for example, "bazel windows git_repository rm command not found" and was unable to find anything relevant.

Any other information, logs, or outputs that you want to share?

If I had to guess, my guess would be that the way MSys bash is being invoked does not set up the environment properly, so only shell built-ins are available. It is not obvious to me what the best way to resolve this issue is.

[irengrig]

Hi @jchv,

seems that you are missing the rm package in your MSYS installation:
/usr/bin/bash: line 4: rm: command not found

There may be possible to install it using MSYS package management: https://github.com/msys2/msys2/wiki/MSYS2-installation.

Hope it helps.

[laszlocsomor]

@jchv, the problem is that the Git repo rule doesn't work well on Windows. For example it looks for Bash in the BAZEL_SH environment variable, and if not found, silently falls back to "bash" (i.e. hoping it's on the PATH).
As a workaround, try setting the BAZEL_SH envvar similar to step 2 in this doc: https://docs.bazel.build/versions/master/install-compile-source.html#windows-1

[laszlocsomor]

Sorry, just realized that that example shows how to set BAZEL_SH from an MSYS window.
If you are using PowerShell, you will need a different syntax. (I don't know.)

In cmd.exe you would do this:

set BAZEL_SH=c:\tools\msys64\usr\bin\bash.exe

(or whatever is the path to your MSYS bash)

[laszlocsomor]

I think the correct solution would be to update the Git repo rule not to depend on Bash:

• obtain the git binary's path from the PATH
• use ctx.execute directly with the git binary, instead of with the bash binary
• implement common Bash functionality (rm, cp) in the repository_ctx object and do those operations as Starlark calls

Yes, this means every git command would need a separate ctx.execute invocation, bloating the code. But currently I'm not aware of a better platform-independent way to run commands in repository rules (without depending on Bash).

[jchv]

Just FWIW, the problem is not really that Bash could not be found - it in fact does work, and it is executing MSys2 bash. The trouble is that my %PATH% does not include MSys2's bin folder, thus meaning that rm is not found - /usr/bin/ is not in the $PATH that MSys2 uses.

I think not depending on Bash is indeed a much better approach.

[ittaiz]

Will there be performance implications for doing this as multiple
consecutive actions?
I have a hunch it will (without concrete data) and external repo cloning is
already an expensive path on our CI
On Thu, 11 Oct 2018 at 23:51 John Chadwick [e-mail] wrote:

Just FWIW, the problem is not really that Bash could not be found - it in
fact does work, and it is executing MSys2 bash. The trouble is that my
%PATH% does not include MSys2's bin folder, thus meaning that rm is not
found - /usr/bin/ is not in the $PATH that MSys2 uses.

I think not depending on Bash is indeed a much better approach.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/6339#issuecomment-429113940,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIFx7DbuHNhnMJtC2hBLdPXP6Ghuftks5uj69AgaJpZM4XSDyl
.

[laszlocsomor]

@ittaiz , I shared your concerns, especially for Windows, but in fact it's already executing consecutive actions -- each git invocation creates a separate process. On Linux that's a fork/exec, which is relatively fast, but on Windows it's a CreateProcess, which is relatively slow.

[laszlocsomor]

To rephrase this bug: this is a feature request for git_repository to work on Windows without needing <msysroot>\usr\bin on the PATH and/or MSYS at all.

[laszlocsomor]

/cc @laurentlb : WDYT about this:

• implement common Bash functionality (rm, cp) in the repository_ctx object and do those operations as Starlark calls

I understand this has implications for local vs. remote execution (in case repo rules must run remotely too) and may require implementing these small utilities as standalone binaries that we ship with Bazel.

[laurentlb]

/cc @aehlig

[jchv]

Hope this isn't a useless question, but at that point would it not make more sense to just build a binary to replace the existing inline bash script instead?

[laszlocsomor]

Hope this isn't a useless question

Not at all. That could be the right solution.
Whoever takes up working on this problem, should consider both approaches.

[filipesilva]

Heya, I was running into this problem as well with rm being invoked directly for the git rule shell script.

@laszlocsomor workaround of adding the C:\mingw64\usr\bin directory to PATH got me through the missing rm but then I hit a missing patch on the http rule:

Starting local Bazel server and connecting to it...
INFO: SHA256 (https://github.com/bazelbuild/buildtools/archive/0.19.2.zip) = d7b05d906b705c8ba4f3caaca2659775d3fbb2db5b4172e9a473a7584fb5d516
INFO: SHA256 (https://codeload.github.com/golang/tools/zip/7b71b077e1f4a3d5f15ca417a16c3b4dbb629b8b) = fe9489eabcb598e13137d0641525ff3813d8af151e1418e6940e611850d90136
ERROR: Analysis of target '//examples/testing:testing_firefox-local' failed; build aborted: no such package '@org_golang_x_tools//go/gcexportdata': Traceback (most recent call last):
        File "C:/users/kamik/_bazel_kamik/5c2rzegw/external/bazel_tools/tools/build_defs/repo/http.bzl", line 55
                patch(ctx)
        File "C:/users/kamik/_bazel_kamik/5c2rzegw/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 80, in patch
                fail(("Error applying patch %s:\n%s%s...)))
Error applying patch @io_bazel_rules_go//third_party:org_golang_x_tools-gazelle.patch:
/usr/bin/bash: patch: command not found
INFO: Elapsed time: 13.128s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (781 packages loaded)
FAILED: Build did NOT complete successfully (781 packages loaded)

patch is not part of the default MSYS2 install. It can be installed, together with other commonly utilities used in Bazel, by running $ pacman -S zip unzip tar perl wget curl diffutils patch in the MSYS2 shell.

This pacman command isn't part of the windows install guide but perhaps should be.

It's worth mentioning that whenever a rule fails to use native.sh_binary, it triggers a cascade of problems:

• binaries like rm and patch are looked up in the system PATH instead of the BAZEL_SH path.
• if they are found in another MSYS2 install you get a This problem is probably due to using incompatible versions of the cygwin DLL. error (https://github.com/bazelbuild/bazel/issues/5724).
• it's very common to have another MSYS2 install by using Git Bash (aka Git for Windows), but it isn't a suitable MSYS2 install because it doesn't have zip nor pacman to get any needed packages.
• to avoid Git Bash dll errors, you have to start a brand new cmd.exe shell. It can't be started from within Git Bash because it will then inherit the PATH.
• starting a new cmd.exe often means dropping out of your editor and working directory to CD back into it.
• after running the initial builds in cmd.exe you can get back to your previous shell, and it will appear to work correctly. But it is likely you'll have to do the whole process again next time the repositories change.

@laszlocsomor mentioned that using --experimental_strict_action_env should avoid the DLL error which means the correct errors should be through even from git bash, but that seems to not always work (https://github.com/bazelbuild/bazel/issues/6759).

[jin]

@irengrig could you add a priority to this issue, please?

[benjamin-bader]

Hi folks, any update on a timeline or a priority for this issue?

(thanks for all your hard work on Bazel - it's a fantastic piece of software)

[irengrig]

This should have been fixed by https://github.com/bazelbuild/bazel/pull/8677, it is going to be released in 0.28.