Rules_go: Having GRPC as a dependency after calling `go_rules_dependencies()` on windows causes failure

[mason-bially]

What version of rules_go are you using?

0.18.5

What version of gazelle are you using?

N/A

What version of Bazel are you using?

0.26.0

Does this issue reproduce with the latest releases of all the above?

Yes

What operating system and processor architecture are you using?

Windows / x86-64

Any other potentially useful information about your toolchain?

Windows C++ 10.0.17763.0

What did you do?

Build a project using GRPC after calling go_rules_dependencies() in my WORKSPACE file.

What did you expect to see?

A successful build.

What did you see instead?

/_bazel_user/hwwnc6kc/external/net_zlib/BUILD.bazel:32:1: C++ compilation of rule '@net_zlib//:zlib' failed (Exit 2)
cl : Command line error D8021 : invalid numeric argument '/Wno-unused-variable'

[mason-bially]

I believe it's this section of code: https://github.com/bazelbuild/rules_go/blame/master/go/private/repositories.bzl#L87

Which has apparently been corrected:
https://github.com/protocolbuffers/protobuf/issues/5472

[jayconrod]

MSVC is not supported with rules_go. The Go toolchain requires GCC or clang in order to build cgo code.

You may find these instructions for setting up msys and MinGW useful. We're working on improving Windows C/C++ toolchain selection, but that probably won't help until Bazel 0.27.0 at least.

It is surprising that net_zlib doesn't compile since it doesn't have anything to do with Go, but this isn't something I can support.

[mason-bially]

I guess I was more asking for including rules_go in my workspace file to not break other dependencies. I'm not expecting it to work on Windows necessarily, I'm expecting it to not break GRPC on Windows by being included.

Bazel is a multi-platform multi-language multi-toolchain build system. My issue is that rules_go is being a bad citizen by modifying other packages in such a way that they break on platforms they support but are unsupported by rules_go.

[mason-bially]

@jayconrod

To reiterate my issue is not with go_rules not working on Windows. Read my issue carefully. My problem is that using go_rules in a workspace file breaks GRPC on Windows.

[jayconrod]

@mason-bially Could you give a more complete example? It would be helpful to have a WORKSPACE file and a bazel build command with a specific target that reproduces the problem.

Simply calling go_rules_dependencies and building Go targets without proto dependencies seems to work fine. I'm also able to build go_proto_libraries with services using MinGW as described above.

[mason-bially]

repro.zip

See attached. Run bazel build exproto with an MSVC configured toolchain to see error. Comment out go_rules_dependencies() to see error go away.

Note that exproto does not depend on go at all, and there is no attempt to build anything go related.

[mason-bially]

@jayconrod Again my problem is that calling go_rules_dependencies() in the WORKSPACE file is modifying other package's dependencies that would otherwise function correctly. If you don't support a platform that's fine, but don't break that platform for the entire workspace! Like, "first, do no harm" or something.

[mason-bially]

The reason for this is that we would like to use go for one of our new services, but we have many projects (>100) managed by our bazel build. Some of them build for windows, some of them rely on the MSVC toolchain. If we can't include go in our workspace file without breaking dozens of projects then we aren't using go. We have no go code, because simply attempting to add it to our build breaks multiple projects.

[jayconrod]

From the original issue, it wasn't clear this was a different gRPC than org_golang_google_grpc. It sounded like you were building something in Go that was intended to work but not with MSVC.

Here's what I think is happening without going too deep into the investigation. net_zlib is (was?) a dependency of com_google_protobuf, which is used internally by proto_library and everything based on it.

go_rules_dependencies declares a dependency on com_google_protobuf (otherwise, go_proto_library would not work). In the version of com_google_protobuf that was current when rules_go 0.18.0 was tagged, com_google_protobuf had some strange dependency issues. They had just introduced a WORKSPACE macro, similar to go_rules_dependencies, which declared zlib and bound net_zlib. I inlined that into go_rules_dependencies to avoid making users call a separate function. People are free to override versions of anything declared in go_rules_dependencies by declaring those repositories before calling go_rules_dependencies, by the way.

I'm guessing what has happened is that a different version of com_google_protobuf is declared somewhere, and it has different dependencies than go_rules_dependencies is prepared to deal with.

This may get more complicated than go_rules_dependencies can deal with. I'd prefer to avoid changing the WORKSPACE boilerplate for using Go, but that might be necessary if there are lots of transitive dependencies.

I'll reopen issue and find some resolution before the next major release, which should be in a few weeks.

[mason-bially]

This works for us, thank you (I may attempt to declare an empty net_zlib to fix it as well, but a local patch for developers working on the go project will suffice for a few weeks).

[jayconrod]

Fixed in #2126.

go_rules_dependencies will no longer declare com_google_protobuf at all. It will need to be declared separately, and its dependency macro will need to be called in WORKSPACE. I don't think there's any way for go_rules_dependencies to declare the repo, then load and call the macro. Inlining the macro makes it difficult to use different versions.

I've also removed org_golang_google_grpc and a few other repositories. Newer versions have too many dependencies to manage in go_rules_dependencies. Since these repositories didn't require any patching, it's best to import these with gazelle update-repos. An example is in gRPC dependencies.

com_github_golang_protobuf, go_googleapis and a few other repositories are still declared in go_rules_dependencies, since they need some additional patching. Hopefully this will be less necessary over time.