Rules_go: Gazelle generated cgo library unable to resolve mac libraries

[emcfarlane]

CGo libraries fail to link frameworks. Given // #cgo LDFLAGS: -framework IOKit the following error is generated:

ERROR: /project/vendor/github.com/shirou/gopsutil/host/BUILD.bazel:3:1: in linkopts attribute of cc_library rule //vendor/github.com/shirou/gopsutil/host:cgo_default_library.cgo_c_lib: could not resolve label 'IOKit'. Since this rule was created by the macro 'cgo_library', the error might have been caused by the macro implementation in /private/var/tmp/_bazel_edward/ad32b996cb424fa4cc44a5c86220cb2f/external/io_bazel_rules_go/go/private/cgo.bzl:402:18.

cgo_library( name = "cgo_default_library", srcs = select({ "@io_bazel_rules_go//go/platform:darwin_amd64": [ "host_darwin_cgo.go", ], "//conditions:default": [], }), clinkopts = select({ "@io_bazel_rules_go//go/platform:darwin_amd64": [ "-framework", "IOKit", ], "//conditions:default": [], }), visibility = ["//visibility:private"], )

[ianthehat]

Not sure if gazelle or the rules are at fault here, possibly both.

  clinkopts = ["-framework IOKit"]

works, whereas

  clinkopts = ["-framework", "IOKit"]

does not, which means there is something that gazelle could generate that would work, but I am not sure why. It seems like the message is coming from bazel not the compiler, so it must be trying to resolve that argument as a file label for some reason, probably in the weird code that tries to decide how to remap linkopts arguments.
I forgot to try
clinkopts = ["-framework=IOKit"]
which would be my ideal...

[jayconrod]

I was experimenting a bit with this yesterday. The error message does make it sound like Bazel is treating the arguments as labels, but I couldn't reproduce this in a separate rule. Continuing to investigate today.

[jayconrod]

This error is the result of a few different things interacting badly. I don't think there's currently any way to correctly specify these flags in cgo directives that will result in a correct BUILD file.

The Bazel documentation says that cc_library linkopts are subject to "Make" variable substitution, Bourne shell tokenization, and label expansion. So unless an option starts with '-' or '$', Bazel will try to resolve it as a label. cgo_library builds a cc_library with these options, so that's why we're seeing this error. Since Bourne shell tokenization is present, we need to pass in linkopts = ["-framework IOKit"] (unfortunately linkopts = ["-framework=IOKit"] does not work).

The portion of go/build related to cgo does not allow spaces in options in cgo directives. See expandSrcDir. Within each option, ${SRCDIR} may contain spaces, but nothing else can. So if you compiled the snippet below, go/build rejects it. Gazelle borrows this code from go/build, so it has the same behavior.

/*
#cgo LDFLAGS: "-framework IOKit"
*/
import "C"

As much as I hate to add a special case, I think we need one here. Gazelle should look for -framework options and join them with the following options.

[jayconrod]

I take it back: no special case needed. Gazelle can just pack all the flags on a line into the same string. Bazel will split them into separate arguments anyway.

[emcfarlane]

@ianthehat @jayconrod thanks for the quick fix!

[emcfarlane]

Gazelle will generate build files as

    clinkopts = select({
        "@io_bazel_rules_go//go/platform:darwin_amd64": [
            "-framework path/to/golib/IOKit",
        ],
        "//conditions:default": [],
    }),

where path/to/golib is the bazel path. Manually fixing to "-framework IOKit", still does not build with fatal error: 'include/smc.c' file not found. for the includes.

[jayconrod]

Reproduced. I think this was caused by #737. Will fix shortly.

[emcfarlane]

Gazelle is bundling clinkopts to a single line,

    clinkopts = select({
        "@io_bazel_rules_go//go/platform:darwin_amd64": [
            "-framework CoreFoundation -framework Security",
        ],

which fails with BUILD.bazel:3:1: in copts attribute of cc_binary rule //vendor/github.com/google/certificate-transparency-go/x509:go_default_library._cgo_.o: each item in the list should contain only one option.

[emcfarlane]

Also have this error from clang, clang: error: unknown argument: '-no-pie'. Issue is talked about here when building go: https://github.com/golang/go/issues/21042 with the fix https://go-review.googlesource.com/c/go/+/49711/3/src/cmd/go/build.go to use -nopie on mac.

Edit: This issues is related to a fix for https://github.com/bazelbuild/rules_go/issues/252

[emcfarlane]

From master go_library with cgo enabled looks like it doesn't pass down clinkopts.

@ianthehat could array initialisation here be an issue: https://github.com/bazelbuild/rules_go/blob/master/go/private/rules/wrappers.bzl#L20 .

[jayconrod]

I'll look into clinkopts being passed through. I thought we had a test for that, so I'm surprised it's not working.

About the copts error (each item in the list should contain only one option): are you sure that's an error and not a warning? I filed bazelbuild/bazel#3526 to remove the warning a while back, but it's still open.

[jayconrod]

Confirmed both issues are bugs.

For copts, we pass these flags to cc_library and cc_binary (to build the code that cgo generates), but we also pass them to cgo itself. cc_library and cc_binary do Bourne shell tokenization on copts plus some other stuff. But cgo and clang do not. So we either need to implement Bourne shell tokenization in the cgo builder (possibly error-prone) or we need to invoke cgo from the builder via the shell (not portable).

For clinkopts, we link the cc_library into a .o file in _cgo_object. This .o file is passed to the final linker, but the flags are not propagated. So we should bubble that up through a provider and pass it to the external linker in go_binary and go_test.

[ains]

Hey @jayconrod bumping into this issue when trying to build https://github.com/shirou/gopsutil on OSX as a vendored dependency with Bazel.

Current error log looks like

INFO: Found 4 targets...
ERROR: /Users/ains/Projects/infra-go/go/src/improbable.io/vendor/github.com/shirou/gopsutil/host/BUILD.bazel:3:1: CGoCodeGen //go/src/improbable.io/vendor/github.com/shirou/gopsutil/host:go_default_library.cgo_codegen failed (Exit 1).
/private/var/tmp/_bazel_ains/d5881538c6f49328692a7434d44d9eb5/bazel-sandbox/2182892111453255433/execroot/__main__/go/src/improbable.io/vendor/github.com/shirou/gopsutil/host/host_darwin_cgo.go:7:11: fatal error: 'include/smc.c' file not found
 #include "include/smc.c"
          ^~~~~~~~~~~~~~~
1 error generated.
CgoCodegen: error running cgo: exit status 1
INFO: Elapsed time: 5.134s, Critical Path: 2.82s

Generated BUILD file in our vendor directory is

load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test")

go_library(
    name = "go_default_library",
    srcs = [
        "host.go",
        "host_fallback.go",
    ] + select({
        "@io_bazel_rules_go//go/platform:darwin_amd64": [
            "host_darwin.go",
            "host_darwin_amd64.go",
            "host_darwin_cgo.go",
        ],
        "@io_bazel_rules_go//go/platform:linux_amd64": [
            "host_linux.go",
            "host_linux_amd64.go",
        ],
        "@io_bazel_rules_go//go/platform:windows_amd64": [
            "host_windows.go",
        ],
        "//conditions:default": [],
    }),
    cgo = True,
    clinkopts = select({
        "@io_bazel_rules_go//go/platform:darwin_amd64": [
            "-framework IOKit",
        ],
        "//conditions:default": [],
    }),
    importpath = "github.com/shirou/gopsutil/host",
    visibility = ["//visibility:public"],
    deps = [
        "//go/src/improbable.io/vendor/github.com/shirou/gopsutil/internal/common:go_default_library",
    ] + select({
        "@io_bazel_rules_go//go/platform:darwin_amd64": [
            "//go/src/improbable.io/vendor/github.com/shirou/gopsutil/process:go_default_library",
        ],
        "@io_bazel_rules_go//go/platform:windows_amd64": [
            "//go/src/improbable.io/vendor/github.com/StackExchange/wmi:go_default_library",
            "//go/src/improbable.io/vendor/github.com/shirou/gopsutil/process:go_default_library",
            "//go/src/improbable.io/vendor/golang.org/x/sys/windows:go_default_library",
        ],
        "//conditions:default": [],
    }),
)

go_test(
    name = "go_default_test",
    srcs = [
        "host_test.go",
    ] + select({
        "@io_bazel_rules_go//go/platform:linux_amd64": [
            "host_linux_test.go",
        ],
        "//conditions:default": [],
    }),
    importpath = "github.com/shirou/gopsutil/host",
    library = ":go_default_library",
)

[emcfarlane]

The package is missing the include dir. Fails with

.../vendor/github.com/shirou/gopsutil/host/host_darwin_cgo.go:7:11: fatal error: 'include/smc.c' file not found
 #include "include/smc.c"
          ^~~~~~~~~~~~~~~
1 error generated.
CgoCodegen: error running cgo: exit status 1
Target //vendor/github.com/shirou/gopsutil/host:go_default_library failed to build

Wierdly adding "include/smc.c", "include/smc.h", to srcs fails with

.../vendor/github.com/shirou/gopsutil/host/host_darwin_cgo.go:7:11: fatal error: 'include/smc.c' file not found
 #include "include/smc.c"
          ^~~~~~~~~~~~~~~
1 error generated.
Target //vendor/github.com/shirou/gopsutil/host:go_default_library failed to build

which is identical apart forom CgoCodegen and has nicer output formating; green highlighting error, bold font on path. @jayconrod is there a better way to import the .c, .h files to satisfy https://github.com/shirou/gopsutil/blob/master/host/host_darwin_cgo.go#L7 . Adding a cc_library build fine but fails on adding to go_library cdeps with

bazel-out/darwin_x86_64-fastbuild/bin/vendor/github.com/shirou/gopsutil/host/go_default_library.cgo_codegen~/vendor_github.com_shirou_gopsutil_host_include_smc.cgo1.c:39:10: fatal error: 'smc.h' file not found
#include "smc.h"
         ^~~~~~~
1 error generated.
Target //vendor/github.com/shirou/gopsutil/host:go_default_library failed to build

[jayconrod]

@ains @afking Sorry for taking so long to look at this again. It fell off my radar after BazelCon.

I got this to build with a few modifications.

• In host_darwin_cgo.go, I removed the line #include "include/smc.c".
• I put the smc files in a separate library.

cc_library(
    name = "smc",
    srcs = ["include/smc.c"],
    hdrs = ["include/smc.h"],
    visibility = ["//visibility:private"],
)

• I added that library to cdeps.

go_library(
    name = "go_default_library",
    ...
    cdeps = select({
        "@io_bazel_rules_go//go/platform:darwin": [
            ":smc",
        ],
    }),

Gazelle is not going to be able to generate this. I'm kind of surprised this works with "go build".

A much better (but more invasive) option would be to move include/smc.h and include/smc.c into the host directory and put // +build darwin at the top of smc.c. That should work with no modification in both "go build" and Bazel and would avoid the need to include the .c file.

[emcfarlane]

@jayconrod thanks!