Rules_go: Bazel 0.26.0 fixed the autoconfigured C++ toolchains and now break cross-compilation of Go

[hlopko]

Hello :)

With the fix for https://github.com/bazelbuild/bazel/issues/8330 we discovered one flaw in our design of Go cross compilation. Currently rules_go depend on C++ toolchain in //go/private:context.bzl (toolchains = ["@bazel_tools//tools/cpp:toolchain_type"]). This dependency exists no matter if we actually have C++ dependencies or not.

The problem is that with https://github.com/bazelbuild/bazel/issues/8330 C++ autoconfigured toolchains properly report their constraints, and now Bazel detects that we don't actually have a C++ cross-compilation toolchain available. As a result the cross-compilation build of go fails:

ERROR: While resolving toolchains for target @io_bazel_rules_go//:go_context_data: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type
ERROR: Analysis of target '//:main' failed; build aborted: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type

To reproduce, create a hello world go_binary, and build it with bazel build //:hello_world --platforms=@io_bazel_rules_go//go/toolchain:darwin_amd64 on linux, (or if you're on mac, use a different platform).

[hlopko]

CC @katre, @philwo

[jayconrod]

@hlopko @katre Do you have a solution for this in mind already? Is there any way to make a toolchain dependency optional?

I think we could define a rule that requires the C++ toolchain. Go rules could depend on that target or not via a select. WDYT?

[hlopko]

I didn't have time to think about it too much.

I guess the simple workaround for the meantime is to revert https://github.com/bazelbuild/rules_go/commit/dd527c7d01af29fa7aabba8860c1ad3ded785153 (I didn't try it yet but it should help).

The separate rule approach is worth thinking about. What will you select on? Ideally you want to take C++ toolchain if its present...

@lberki had a nice evil idea - have an aspect traverse go dependencies, and if it finds C++ target, it steals its C++ toolchain from it. There's a lot of handwaving there :))

[hlopko]

Another not great alternative is to register Null-object-like C++ toolchains that can only fail the build when actually used.

[jayconrod]

Would reverting dd527c7 break cgo compilation in general? Is that required in Bazel 0.26.0, or still optional?

About selecting on a separate rule, I think this will be easier once SBC is fully enabled. We'll probably have a control whether cgo is enabled, and we could select on that.

Another idea: we could define a separate set of go_toolchains that depend on the C++ toolchain, then add constraint_setting / constraint_values and distinct cgo platforms. Rules would only require the Go toolchain directly.

[lberki]

The "null toolchain" would only help if you could make it so that it's selected iff nothing else is. I thought that was not possible.

[hlopko]

Reverting dd527c7 will make go cross compilation work if there are no C++ dependencies. It will not make cgo with C++ dependencies work, cgo would only cross compile Go code, not the C++ code.

Having a cgo option is a solution. This way users have to specify that they intend to use cgo and go rules will only then depend on C++ toolchain (and complain when cross compiling that the matching C++ toolchain is not present). I think SBC is non-experimental in 0.27, but we should double check.

The last approach you propose is to have distinct sets of platforms for go and cgo, right? That will work as well, it's similar to the previous solution, except that we select on constraint_value, not on configuration option.

Is that reasonable thing to ask from Go users to be explicit about cgo usage? Or they expect this to work automatically?

[katre]

I've kicked around the idea of having optional toolchains, in addition to
required toolchains. If we had that, then go could declare that the cpp
toolchain is optional, and if trying to build cgo code without a cpp
toolchain, produce its own error.

The code paths wouldn't be particularly tricky to add (but would require
starlark changes in the rule declaration API, and lots of tests). Does this
sound like a reasonable way forward?

On Wed, Jun 5, 2019 at 4:58 AM Marcel Hlopko [e-mail]
wrote:

Reverting dd527c7
https://github.com/bazelbuild/rules_go/commit/dd527c7d01af29fa7aabba8860c1ad3ded785153
will make go cross compilation work if there are no C++ dependencies. It
will not make cgo with C++ dependencies work, cgo would only cross compile
Go code, not the C++ code.

Having a cgo option is a solution. This way users have to specify that
they intend to use cgo and go rules will only then depend on C++ toolchain
(and complain when cross compiling that the matching C++ toolchain is not
present). I think XBC is non-experimental in 0.27, but we should double
check.

The last approach you propose is to have distinct sets of platforms for go
and cgo, right? That will work as well, it's similar to the previous
solution, except that we select on constraint_value, not on configuration
option.

Is that reasonable thing to ask from Go users to be explicit about cgo
usage? Or they expect this to work automatically?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/rules_go/issues/2089?email_source=notifications&email_token=AACPW75LNOMREIJTP7QNXELPY553VA5CNFSM4HS5I7P2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW7CCKY#issuecomment-498999595,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACPW75OSDHWRPPF3C4PELTPY553VANCNFSM4HS5I7PQ
.

[lberki]

Shouting from sidelines: I could imagine conditional toolchains (i.e. "I need this toolchain in some cases, but not in others") but optional (as in "it's useful if I have it, but I can deal with it it's not available") is weird.

Let's try to keep the user-facing API simple because the API is something everyone who uses toolchains needs to understand and they don't benefit from something added only for Go. If Go is the only place where a new API feature would be useful, I'd prefer the hackish "fish the C++ toolchain out from transitive dependencies" approach than extending the API. That way, it's only Go who pays the price.

[katre]

I am trying to remember who, besides Go, asked for this feature. The actual
feature was requested by Ian for rules_go (
https://github.com/bazelbuild/bazel/issues/3601) about two years ago.

It hasn't come up much since, but other toolchains have asked about this
before.

On Wed, Jun 5, 2019 at 10:24 AM lberki [e-mail] wrote:

Shouting from sidelines: I could imagine conditional toolchains (i.e. "I
need this toolchain in some cases, but not in others") but optional (as in
"it's useful if I have it, but I can deal with it it's not available") is
weird.

Let's try to keep the user-facing API simple because the API is something
everyone who uses toolchains needs to understand and they don't necessarily
benefit from something added only for Go. If Go is the only place where a
new API feature would be useful, I'd prefer the hackish "fish the C++
toolchain out from transitive dependencies" approach than extending the
API. That way, it's only Go who pays the price.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/rules_go/issues/2089?email_source=notifications&email_token=AACPW72ORLH4YATSNYBXA4DPY7ECDA5CNFSM4HS5I7P2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODW735FA#issuecomment-499105428,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACPW72BFZWIBPLR6A6CZQTPY7ECDANCNFSM4HS5I7PQ
.

[jayconrod]

Is that reasonable thing to ask from Go users to be explicit about cgo usage? Or they expect this to work automatically?

I think it's reasonable. With the Go command, cgo is enabled by default if the target platform is the same as the host platform and disabled otherwise. Users can explicitly turn it on by setting CGO_ENABLED=1 in their environment.

So here's what I'm thinking now:

• rules_go would define a cgo constraint_setting with two values: cgo_on and cgo_off. The default is cgo_on.
• Platforms defined in rules_go will have two variants, one for each cgo mode. So you could build with --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64_cgo to get cross-compilation with cgo, and --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 without cgo.
  
  
  
  • This shouldn't break anyone. Currently, all Go toolchains require cgo, but we don't enable it internally by default when cross-compiling since people almost never have a working cross-compiling C/C++ toolchain set up.
  
  
• We'll define a toolchain for each platform. Each cgo_on variant will depend on a target that requests a @bazel_tools//tools/cpp:toolchain_type toolchain. Each cgo_off variant will not depend on this target.

[hlopko]

If CGO_ENABLED is how people configure cgo outside of Bazel, then let's do this!

[jayconrod]

This should be fixed in #2090.

• On the default platform, cgo is enabled.
• With --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64, cgo is disabled, and no C/C++ toolchain is required. Previously, cgo was disabled, but we always required a C/C++ toolchain, even though we didn't use it.
• With --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64_cgo, cgo is enabled, and a compatible C/C++ toolchain is required. These platforms are new.

cc @steeve

@hlopko Any update on the mingw constraint being required by the auto-configured C/C++ toolchains on Windows?

Unfortunately, I can't think of a way to get cgo to work on Windows by default without requiring users to configure their own toolchains or platform since MSVC is always selected. That was already a problem though (they had to say --cpu=x64_windows --compiler=mingw-gcc).

[asv]

I have problems with cross-compiling (windows_amd64 under linux) even in commit f2373c9fbd09586d8e591dda3c43d66445b2d7ca...

bazel: 0.26.1
rules_go: f2373c9fbd09586d8e591dda3c43d66445b2d7ca

$ bazel build --platforms=@io_bazel_rules_go//go/toolchain:windows_amd64 //cmd/...
ERROR: /home/asmirnov/.cache/bazel/_bazel_asmirnov/716d67f5e74ab6acaf0df7cd1e3df431/external/io_bazel_rules_go/proto/wkt/BUILD.bazel:3:1: in go_proto_library rule @io_bazel_rules_go//proto/wkt:wrappers_go_proto: 
Traceback (most recent call last):
        File "/home/asmirnov/.cache/bazel/_bazel_asmirnov/716d67f5e74ab6acaf0df7cd1e3df431/external/io_bazel_rules_go/proto/wkt/BUILD.bazel", line 3
                go_proto_library(name = 'wrappers_go_proto')
        File "/home/asmirnov/.cache/bazel/_bazel_asmirnov/716d67f5e74ab6acaf0df7cd1e3df431/external/io_bazel_rules_go/proto/def.bzl", line 116, in _go_proto_library_impl
                go_srcs.extend(compiler.compile(go, compiler = ...))
        File "/home/asmirnov/.cache/bazel/_bazel_asmirnov/716d67f5e74ab6acaf0df7cd1e3df431/external/io_bazel_rules_go/proto/def.bzl", line 116, in go_srcs.extend
                compiler.compile(go, compiler = compiler, protos = [g...], <2 more arguments>)
        File "/home/asmirnov/.cache/bazel/_bazel_asmirnov/716d67f5e74ab6acaf0df7cd1e3df431/external/io_bazel_rules_go/proto/compiler.bzl", line 72, in compiler.compile
                args.add("-compiler_path", go.cgo_tools.c_c...])
        File "/home/asmirnov/.cache/bazel/_bazel_asmirnov/716d67f5e74ab6acaf0df7cd1e3df431/external/io_bazel_rules_go/proto/compiler.bzl", line 72, in args.add
                go.cgo_tools.c_compiler_path
object of type 'NoneType' has no field 'c_compiler_path'

Is there a working workaround, if I do not need support cgo?

[jayconrod]

@asv We shouldn't need the C compiler to run protoc. I'll look into that. We're using its path to set a value for PATH, but I don't think that actually makes sense.

--platforms=@io_bazel_rules_go//go/toolchain:windows_amd64_cgo may work if you have configured a working C/C++ cross compiler.

[steeve]

I have the same problem as https://github.com/bazelbuild/rules_go/issues/2089#issuecomment-502983312 when I'm using a starlark transition to set the correct platform based on the current cpu (where go is a dependency of a ios_framework or android_binary that's multiarch).

My transition:

_ANDROID_CPUS_TO_PLATFORMS = {
    "x86": "@io_bazel_rules_go//go/toolchain:android_386_cgo",
    "x86_64": "@io_bazel_rules_go//go/toolchain:android_amd64_cgo",
    "armeabi-v7a": "@io_bazel_rules_go//go/toolchain:android_arm_cgo",
    "arm64-v8a": "@io_bazel_rules_go//go/toolchain:android_arm64_cgo",
}

_IOS_CPUS_TO_PLATFORMS = {
    "ios_armv7": "@io_bazel_rules_go//go/toolchain:ios_arm_cgo",
    "ios_arm64": "@io_bazel_rules_go//go/toolchain:ios_arm64_cgo",
    "ios_i386": "@io_bazel_rules_go//go/toolchain:ios_386_cgo",
    "ios_x86_64": "@io_bazel_rules_go//go/toolchain:ios_amd64_cgo",
}

def _go_platform_transition_impl(settings, attr):
    cpu = settings["//command_line_option:cpu"]
    platform = ""
    if settings["//command_line_option:crosstool_top"].workspace_name == "androidndk":
        platform = _ANDROID_CPUS_TO_PLATFORMS[cpu]
    elif cpu in _IOS_CPUS_TO_PLATFORMS:
        platform = _IOS_CPUS_TO_PLATFORMS[cpu]
    return {
        "//command_line_option:platforms": platform,
    }

go_platform_transition = transition(
    implementation = _go_platform_transition_impl,
    inputs = [
        "//command_line_option:cpu",
        "//command_line_option:crosstool_top",
    ],
    outputs = [
        "//command_line_option:platforms",
    ],
)

On android the error is somewhat different:

ERROR: While resolving toolchains for target //examples/helloworld:HelloLib.java.cc_library: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type

[steeve]

If I enable legacy objc/cgo mode:

ERROR: /Users/steeve/go/src/github.com/znly/rules_gomobile/examples/helloworld/BUILD.bazel:13:1: in _cgo_codegen rule //examples/helloworld:HelloLib.objc.binary%js_wasm%cgo_codegen:
Traceback (most recent call last):
    File "/Users/steeve/go/src/github.com/znly/rules_gomobile/examples/helloworld/BUILD.bazel", line 13
        _cgo_codegen(name = 'HelloLib.objc.binary%js_wasm%cgo_codegen')
    File "/private/var/tmp/_bazel_steeve/7a2b891e90b0ccfacc379ac7b2b2ca04/external/io_bazel_rules_go/go/private/rules/cgo.bzl", line 269, in _cgo_codegen_impl
        fail("Go toolchain does not support c...")
Go toolchain does not support cgo

[jayconrod]

2101 will remove the go_proto_compiler dependency on the C/C++ toolchain. I don't think we ever really needed it.

However, I'm not able to build anything proto-related on Windows without MSVC, including proto_library. The mingw toolchain doesn't get picked up, even with --host_platform set to a toolchain with the mingw constraint. Not really sure how to proceed. @hlopko any ideas?

[hlopko]

(Sorry, we have a summit this whole week, will try to get to this asap)

[chrislovecnm]

Any updates on this?

[chrislovecnm]

So I configured a project to use the latest commit:

http_archive(
    name = "io_bazel_rules_go",
    urls = ["https://github.com/bazelbuild/rules_go/archive/8ea79bbd5e6ea09dc611c245d1dc09ef7ab7118a.tar.gz"],
    sha256 = "039c159e8f16a365a5f0ce31c8e6ca8acba5708962e904c1f164ba79fb8efe4d",
    strip_prefix = "rules_go-8ea79bbd5e6ea09dc611c245d1dc09ef7ab7118a",
)

Good news is that I am able to build a linux binary on a darwin platform.

$ bazel version
Build label: 0.26.1-homebrew

Bad news is that rules_docker is now having issues loading toolchains. I have filed an issue with them https://github.com/bazelbuild/rules_docker/issues/943.

Seems to be a interop issue with rules_go https://github.com/bazelbuild/rules_go/issues/2115

[munnerz]

I can confirm I have the same issues as @chrislovecnm above when building with the --platforms flag set and the above WORKSPACE rule (8ea79bbd5e6ea09dc611c245d1dc09ef7ab7118a).

bazel run --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 //cmd/acmesolver:image
ERROR: While resolving toolchains for target //cmd/acmesolver:image: no matching toolchains found for types @io_bazel_rules_docker//toolchains/docker:toolchain_type
ERROR: Analysis of target '//cmd/acmesolver:image' failed; build aborted: no matching toolchains found for types @io_bazel_rules_docker//toolchains/docker:toolchain_type
INFO: Elapsed time: 18.499s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (5 packages loaded, 103 targets configured)
FAILED: Build did NOT complete successfully (5 packages loaded, 103 targets configured)

[chrislovecnm]

@munnerz I’m gonna look at this tomorrow with @nlopezgi. I have a separate issue tracking this btw #2115

[strican]

@chrislovecnm, in that case should we close this issue? Or is there more work associated with this issue that isn't covered by #2115?

[chrislovecnm]

@strican I will gracefully bow out of that decision, no idea with we have more toolchain work to accomplish. I opened a separate for the rules_docker issues.

Probably more @hlopko or @jayconrod's call I think.

[hlopko]

Sorry to be slow, the summit took the best of me.

So the C++ rules plan is:

1) allow platforms to contain multiple values for a single constraint_setting (https://github.com/bazelbuild/bazel/issues/8763).
1) create 'compiler' constraint_setting for rules_cc (https://github.com/bazelbuild/rules_cc/issues/24)
1) properly populate 'compiler' in all autoconfigured C++ toolchains (https://github.com/bazelbuild/bazel/issues/8766)
1) and properly populate 'compiler' in Bazel's autogenerated host platform target (https://github.com/bazelbuild/rules_cc/issues/24)

Then rules go will be able to use constraints in their cgo platforms to influence C++ toolchain selection. This will take a while to get all the pieces together.

[steeve]

Still a problem on 0.28.0 and latest master:

ios:

build:ios_arm64         --platforms=@io_bazel_rules_go//go/toolchain:ios_arm64_cgo
build:ios_arm64         --apple_platform_type=ios
build:ios_arm64         --ios_multi_cpus=arm64
build:ios_arm64         --cpu=ios_arm64

$ bazel build //examples/helloworld:HelloLib.framework --config ios_arm64
DEBUG: /private/var/tmp/_bazel_steeve/7a2b891e90b0ccfacc379ac7b2b2ca04/external/io_bazel_rules_go/go/private/context.bzl:468:9: >>>>>>>>>NAN
ERROR: /private/var/tmp/_bazel_steeve/7a2b891e90b0ccfacc379ac7b2b2ca04/external/io_bazel_rules_go/BUILD.bazel:8:1: in @io_bazel_rules_go//go/private:rules/aspect.bzl%go_archive_aspect[pure="off",static="auto",msan="auto",race="auto",goos="auto",goarch="auto",linkmode="c-archive"] aspect on stdlib rule @io_bazel_rules_go//:stdlib:
Traceback (most recent call last):
    File "/private/var/tmp/_bazel_steeve/7a2b891e90b0ccfacc379ac7b2b2ca04/external/io_bazel_rules_go/BUILD.bazel", line 8
        @io_bazel_rules_go//go/private:rules/aspect.bzl%go_archive_aspect(...)
    File "/private/var/tmp/_bazel_steeve/7a2b891e90b0ccfacc379ac7b2b2ca04/external/io_bazel_rules_go/go/private/rules/aspect.bzl", line 61, in _go_archive_aspect_impl
        go.library_to_source(go, ctx.rule.attr, library, ctx.covera...())
    File "/private/var/tmp/_bazel_steeve/7a2b891e90b0ccfacc379ac7b2b2ca04/external/io_bazel_rules_go/go/private/context.bzl", line 267, in go.library_to_source
        library.resolve(go, attr, source, _merge_embed)
    File "/private/var/tmp/_bazel_steeve/7a2b891e90b0ccfacc379ac7b2b2ca04/external/io_bazel_rules_go/go/private/actions/stdlib.bzl", line 42, in library.resolve
        _build_stdlib(go)
    File "/private/var/tmp/_bazel_steeve/7a2b891e90b0ccfacc379ac7b2b2ca04/external/io_bazel_rules_go/go/private/actions/stdlib.bzl", line 72, in _build_stdlib
        env.update({"CGO_ENABLED": "1", "CC": go.cg...))})
    File "/private/var/tmp/_bazel_steeve/7a2b891e90b0ccfacc379ac7b2b2ca04/external/io_bazel_rules_go/go/private/actions/stdlib.bzl", line 74, in env.update
        go.cgo_tools.c_compiler_path
object of type 'NoneType' has no field 'c_compiler_path'

Android:

build:android_arm64     --crosstool_top=@androidndk//:toolchain-libcpp
build:android_arm64     --host_crosstool_top=@bazel_tools//tools/cpp:toolchain
build:android_arm64     --fat_apk_cpu=arm64-v8a
build:android_arm64     --platforms=@io_bazel_rules_go//go/toolchain:android_arm64_cgo
build:android_arm64     --cpu=arm64-v8a

DEBUG: /private/var/tmp/_bazel_steeve/7a2b891e90b0ccfacc379ac7b2b2ca04/external/co_znly_rules_gomobile/platform/transitions.bzl:22:5: >>>>>>> @io_bazel_rules_go//go/toolchain:android_arm64_cgo
ERROR: Analysis of target '//examples/helloworld:HelloApp' failed; build aborted: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type

[steeve]

When disabling the cgo_off platforms, it seems the cc toolchain is not properly detected, I don't see the android one showing up.

[steeve]

I set up a repro at https://github.com/steeve/rules_go_bazel_toolchains

[steeve]

Weird fact, the problem also appears with cc_binary targets. On the repo above:

$ bazel build --config=android_arm64 //:main_cc

This works if I remove the --platforms.

My guess is that the android rules are not platform aware, and that applying @bazel_tools//platforms:android and @bazel_tools//platforms:aarch64 makes it fail.

[steeve]

Enabling toolchain_resolution_debug:

INFO: Build option --crosstool_top has changed, discarding analysis cache.
INFO: ToolchainResolution: Looking for toolchain of type @bazel_tools//tools/cpp:toolchain_type...
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-armeabi-v7a...
INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:arm, which does not match value @platforms//cpu:aarch64 from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-armeabi-v7a, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-darwin_x86_64...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:osx, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:x86_64, which does not match value @platforms//cpu:aarch64 from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-darwin_x86_64, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-ios_arm64...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-ios_arm64, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-ios_arm64e...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-ios_arm64e, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-ios_armv7...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:arm, which does not match value @platforms//cpu:aarch64 from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-ios_armv7, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-ios_i386...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:x86_32, which does not match value @platforms//cpu:aarch64 from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-ios_i386, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-ios_x86_64...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:x86_64, which does not match value @platforms//cpu:aarch64 from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-ios_x86_64, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-tvos_arm64...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-tvos_arm64, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-tvos_x86_64...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:x86_64, which does not match value @platforms//cpu:aarch64 from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-tvos_x86_64, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-watchos_arm64_32...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-watchos_arm64_32, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-watchos_armv7k...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:arm, which does not match value @platforms//cpu:aarch64 from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-watchos_armv7k, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-watchos_i386...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:x86_32, which does not match value @platforms//cpu:aarch64 from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-watchos_i386, because of target platform mismatch
INFO: ToolchainResolution:   Considering toolchain @local_config_cc//:cc-compiler-watchos_x86_64...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:ios, which does not match value @platforms//os:android from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:x86_64, which does not match value @platforms//cpu:aarch64 from the target platform @io_bazel_rules_go//go/toolchain:android_arm64_cgo
INFO: ToolchainResolution:   Rejected toolchain @local_config_cc//:cc-compiler-watchos_x86_64, because of target platform mismatch
INFO: ToolchainResolution:   No toolchains found
ERROR: While resolving toolchains for target //:main_cc: no matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type

This is weird because it's only testing @local_config_cc

[steeve]

Findings:

• turns out the cgo_off toolchain is selected because of the non requirement of cgo when cross compiling
• somehow find_cpp_toolchain returns the host platform instead of the selected cpp toolchain (android)

[steeve]

Here is the weird thing I using debug toolchains:

INFO: ToolchainResolution:   Considering toolchain @androidndk//:aarch64-linux-android-clang7.0.2-libcpp...
INFO: ToolchainResolution:     Toolchain constraint @platforms//os:os has value @platforms//os:android, which does not match value @platforms//os:osx from the target platform @bazel_tools//platforms:target_platform
INFO: ToolchainResolution:     Toolchain constraint @platforms//cpu:cpu has value @platforms//cpu:aarch64, which does not match value @platforms//cpu:x86_64 from the target platform @bazel_tools//platforms:target_platform
INFO: ToolchainResolution:   Rejected toolchain @androidndk//:aarch64-linux-android-clang7.0.2-libcpp, because of target platform mismatch

It's weird because the target is android aarch64

[jayconrod]

@steeve #2140 should print a better error message in this situation. A C/C++ toolchain is needed for linkmode = "c-archive", but nothing gets configured.

I'm not familiar with the details of how these toolchains are declared or registered. The bazel folks will be better able to help. It looks like you've already filed bazelbuild/bazel#8871 to look into this.

[hlopko]

Drive by comment - android rules are not yet prepared for --platforms.

[steeve]

@hlopko I could get away with manually registering them, though:

[
    toolchain(
        name = "android_%s" % cpu,
        target_compatible_with = [
            "@bazel_tools//platforms:android",
            "@bazel_tools//platforms:%s" % cpu,
        ],
        toolchain = "@androidndk//:%s-linux-android-clang7.0.2-libcpp" % cpu,
        toolchain_type = "@bazel_tools//tools/cpp:toolchain_type",
    )
    for cpu in ["x86", "x86_64", "arm", "aarch64"]
]

[
    register_toolchains("//:android_%s" % cpu)
    for cpu in ["x86_64", "arm", "aarch64"]
]

Is there anything more needed? I can compile cc_* rules just fine with --incompatible_enable_cc_toolchain_resolution.

[hlopko]

Registering ndk toolchains is one of the problems. Other is having the fat apk produced correctly.

[steeve]

@hlopko the problem though is that it seems if we put a --platform, then the crosstool_top is ignored (and I guess the transition from android_crosstool_top as well)

[Toxicable]

I believe this causes and issue for people using rules_docker

We're upgrading rules_docker to 0.12.0 which now depends on rules_go and we're getting this error in CI:

ERROR: /home/runner/.cache/bazel/_bazel_runner/f2e96da83c9a9bca36350376aeb4df02/external/io_bazel_rules_go/BUILD.bazel:80:1: Target '@io_bazel_rules_go//:cgo_context_data' depends on toolchain '@local_config_cc//:cc-compiler-k8', which cannot be found: error loading package '@local_config_cc//': Unable to load file '@local_config_cc//:armeabi_cc_toolchain_config.bzl': file doesn't exist'
ERROR: Analysis of target '//apps/APPNAME/be:image' failed; build aborted: error loading package '@local_config_cc//': Unable to load file '@local_config_cc//:armeabi_cc_toolchain_config.bzl': file doesn't exist

We don't need cgo or cross compiling of any C/C++
Is there a way to disable this?
I don't know much about how this is configured, but from reading through here I set: build --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 however that still requires that local_config_cc be configured as the error above shows.

[jayconrod]

Just to recap where we are with this issue:

• In some configurations, rules_go needs a C/C++ toolchain in order to build cgo code. Even if there is no cgo code, a C linker is needed in some configurations. The Go standard library also has some optional C dependencies.
• So what rules_go really needs is an optional C/C++ toolchain dependency that's only provided in some configurations. I've attempted to do that by requiring a C/C++ toolchain from @io_bazel_rules_go//:cgo_context_data, which is a dependency of @io_bazel_rules_go//:go_context_data, hidden behind a select expression which only picks it on platforms where cgo is enabled. go_context_data is a common dependency of all Go rules.
• Bazel seems to analyze cgo_context_data even though it isn't required. bazel cquery --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64 'somepath(@io_bazel_rules_go//tests/core/go_binary:hello, @io_bazel_rules_go//:cgo_context_data)' shows that there's no path through the configured target graph from an example binary rule to cgo_context_data. It's possible there's some dependency sneaking through the toolchains, or there might be a bug in Bazel.

I've spent several weeks analyzing this issue, and I've basically timeboxed it at this point. It should be fixed eventually but probably won't be any time soon. I'd recommend folks work around this by installing a C/C++ toolchain or configuring a dummy cc_toolchain.

[Toxicable]

@jayconrod We have bazel installation doc: https://docs.bazel.build/versions/master/install-ubuntu.html#step-1-install-required-packages but it fails with the same error.
I don't know how else to debug what system deps are required to work around this.
This is the bazel section of our ci container dockerfile, which has a ubuntu:18.04 base:

# Bazel
RUN apt-get update && apt-get -y install --no-install-recommends pkg-config zip g++ zlib1g-dev unzip python3 git \
&& yarn global add @bazel/[email protected] \
&& bazel version

Something else that might be helpful is with: build --toolchain_resolution_debug we see this:

INFO: ToolchainResolution: Selected execution platform @local_config_platform//:host, type @io_bazel_rules_go//go:toolchain -> toolchain @go_sdk//:go_linux_amd64_cgo-impl

I can post the full log output, it's just that it's a couple of a hundred lines.
Is there a flag we can pass to force using the non cgo toolchain?

[jayconrod]

@Toxicable Do you have gcc or clang available in the container? Are you able to build a minimal cc_binary?

When I built a container earlier today following those instructions, something pulled in build-essential, which is probably the package you need. --no-install-recommends may be causing problems.

[Toxicable]

I removed --no-install-recommends
Which I now see build-essential in the list of packages installed, however I did try installing that seprately previously.

In CI

• gcc --help works correctly
• cc_binary works correctly (npx bazel run :hello -> prints out hello after compiling)

But still failing on this error:

ERROR: /home/runner/.cache/bazel/_bazel_runner/f2e96da83c9a9bca36350376aeb4df02/external/io_bazel_rules_go/BUILD.bazel:80:1: Target '@io_bazel_rules_go//:cgo_context_data' depends on toolchain '@local_config_cc//:cc-compiler-k8', which cannot be found: error loading package '@local_config_cc//': Unable to load file '@local_config_cc//:armeabi_cc_toolchain_config.bzl': file doesn't exist'

Note this part here:

Unable to load file '@local_config_cc//:armeabi_cc_toolchain_config.bzl': file doesn't exist'

Whys it looking for arm toolchain configs?

[jayconrod]

@Toxicable rules_go doesn't load that file directly. Something in the Bazel toolchain auto-configuration is probably loading it. Is this in a freshly started container, or was build-essential installed after running a bazel command? If the latter, you may need to run bazel clean --expunge to get it to reconfigure the toolchain.

I don't know enough about your environment to help more than that. Feel free to open a new issue with enough information to reproduce the problem, and I can take a look.

[nlopezgi]

thanks for the additional info @Toxicable , I think the error might be related to some recent toolchain changes we made in rules_docker.
Can you run
bazel query --output=label @local_config_cc//...
Also, can you try to use https://github.com/bazelbuild/rules_docker/pull/1205 to see if that fixes your issue. Apologies for the problem. Toolchains are hard!

[Toxicable]

It's alright, thanks for taking the time to debug this.

Both locally and in CI I get:

f.wiles@fwiles:~/workspace/WORKSPACENAME$ npx bazel query --output=label @local_config_cc//...
INFO: Writing tracer profile to '/home/f.wiles/.cache/bazel/_bazel_f.wiles/4e95cbad76bfb4ed000904165db39268/command.profile.gz'
WARNING: Switched off --watchfs again... temporarily falling back to manually checking files for changes
@local_config_cc//:toolchain
@local_config_cc//:malloc
@local_config_cc//:cc_wrapper
@local_config_cc//:cc-compiler-k8
@local_config_cc//:local
@local_config_cc//:compiler_deps
@local_config_cc//:cc-compiler-armeabi-v7a
@local_config_cc//:stub_armeabi-v7a
@local_config_cc//:empty
Loading: 0 packages loaded

when trying to use your PR I get this error locally:

ERROR: An error occurred during the fetch of repository 'nodejs_base':
   Pull command failed: 2019/10/12 10:51:03 Running the Image Puller to pull images from a Docker Registry...
2019/10/12 10:51:04 Image pull was unsuccessful: reading image "index.docker.io/toxicable/debian9-curl@sha256:bddb0e543b4556a8b9583f7bad36370aaa1e81ebaa8b09ec771268ec15cc69b5": <empty transport.Error response>
 (/home/f.wiles/.cache/bazel/_bazel_f.wiles/4e95cbad76bfb4ed000904165db39268/external/go_puller_linux/file/downloaded -directory /home/f.wiles/.cache/bazel/_bazel_f.wiles/4e95cbad76bfb4ed000904165db39268/external/nodejs_base/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name index.docker.io/toxicable/debian9-curl@sha256:bddb0e543b4556a8b9583f7bad36370aaa1e81ebaa8b09ec771268ec15cc69b5)

Note that nodejs_base is not the default one from ruels_docker, it's one we defiend at index.docker.io/toxicable/debian9-curl

[nlopezgi]

thanks @Toxicable , it looks like that PR solves the toolchain issue, let's open up a separate issue in rules_docker to talk about the issue you are having pulling that image.

[lubinszARM]

Hi,
Any progress of this issue? I found the same problem.
@jayconrod @hlopko

[yicm]

Hi, @jayconrod @hlopko The platform functions of rules_go cannot be unified with those of other languages. For example, I configured the C++ platform tool chain and customized the platform constraints, while rules_go fixed the constraints of the platform functions. For a mixed project of C++ and Go, I need to divide two compilations, is there a way to use the following commands to achieve one-time compilation?

bazel build --platforms=//platforms:custom_platform  //...

[asv]

We also face a similar problem (like @yicm) in a mixed Java project:

$  bazelisk build //... --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64_cgo 
2020/08/05 17:43:05 Downloading https://releases.bazel.build/3.4.1/release/bazel-3.4.1-darwin-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
ERROR: While resolving toolchains for target //java/svc2/src/main/java/com/example/svc2:svc2: No matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type. Maybe --incompatible_use_cc_configure_from_rules_cc has been flipped and there is no default C++ toolchain added in the WORKSPACE file? See https://github.com/bazelbuild/bazel/issues/10134 for details and migration instructions.
ERROR: Analysis of target '//java/svc2/src/main/java/com/example/svc2:svc2' failed; build aborted: No matching toolchains found for types @bazel_tools//tools/cpp:toolchain_type. Maybe --incompatible_use_cc_configure_from_rules_cc has been flipped and there is no default C++ toolchain added in the WORKSPACE file? See https://github.com/bazelbuild/bazel/issues/10134 for details and migration instructions.
INFO: Elapsed time: 52.756s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (14 packages loaded, 110 targets configure

I can create a separate issue along with the repro repository if needed.

Edit: problem occurs only under Mac OS. On linux we have no problem with cross compilation.

[yicm]

Hi, @asv My error message is the same as yours. I think there is a need to create a new issue. I have not succeeded under the Linux platform. Maybe we can show this problem better through a bazel demo project.

[jayconrod]

Closing this issue because it's been fixed for a long time. At the time, rules_go had an unconditional dependency on the C/C++ toolchain in configurations where it wasn't needed and wasn't available.

@asv @yicm Could you open new issues and fill out the issue template? I don't think the issues you're commenting on are related to the problems you're seeing, but I don't have enough information to help more than that.