Rules_go: Dependency on golang.org/x/sys/unix missing in 0.14

[Xjs]

0.14.0 upgraded the google.golang.org/grpc dependency to version 1.14 (https://github.com/bazelbuild/rules_go/commit/c73c25f2ead2b6d2cf772878e88a650f07a3de7c#diff-fdb6ae35486586f3473cd99b5d4c4815R128). However, google.golang.org/grpc introduced a dependency on golang.org/x/sys/unix from 1.13 to 1.14. (Exactly here: https://github.com/grpc/grpc-go/commit/7268ca41d3d23f659b40e084278bede42d6e6d42#diff-be8985446e731a737c227f2b89d6bf1c)

This leads to targets depending on @org_golang_google_grpc//:go_default_library breaking with rules_go 0.14.0. Artificial example: https://github.com/Xjs/protoTest/tree/v0.14.0. (Breaks on linux only for me, works on Windows, interestingly enough.)

Would it break stuff to add golang.org/x/sys/unix to go/private/repositories.bzl? Am I missing something? (I remember seeing issues relating to golang.org/x/sys/unix...)

[jayconrod]

For consistency with the other dependencies (e.g., org_golang_x_net), we should probably declare it.

I'm a bit hesitant to declare this because a lot of people may be depending on specific versions, and providing something within go_rules_dependencies may block that. It is possible to override repositories, but Bazel can ignore overrides for repositories that have already been resolved, so this may cause some confusion.

So... needs investigation, but should be done soon (or not).

[jayconrod]

Okay, after some experimentation, it seems that it is now consistently possible to override rules by calling go_repository before go_rules_dependencies(). That didn't used to be the case because Gazelle had some external dependencies (which are now vendored).

So #1649 will add org_golang_x_sys to go_rules_dependencies. I will likely cherry-pick that to release-0.14 and tag 0.14.1 next week.

1650 adds a FAQ and clarifies the documentation on overriding repository rules.

[krishicks]

I'm running into an issue around golang.org/x/sys/unix with 0.15.3. I've only noticed this failing when cross-compiling from macOS to a Linux target. Is this the same error that was meant to be fixed in 0.14.1 or is it different? Do I need to declare the repository?

INFO: From GoStdlib external/io_bazel_rules_go/linux_amd64_stripped/stdlib%/pkg:
# os/user
bazel-out/darwin-fastbuild/bin/external/io_bazel_rules_go/linux_amd64_stripped/stdlib%/src/os/user/getgrouplist_unix.go:16:35: warning: passing 'gid_t *' (aka 'unsigned int *') to parameter of type 'int *' converts between pointers to integer types with different sign [-Wpointer-sign]
/usr/include/unistd.h:653:43: note: passing argument to parameter here
ERROR: /Users/kris/workspace/lightstep/go/src/github.com/lightstep/vendor/google.golang.org/grpc/internal/channelz/BUILD:3:1: GoCompile vendor/google.golang.org/grpc/internal/channelz/linux_amd64_stripped/go_default_library%/vendor/google.golang.org/grpc/internal/channelz.a failed (Exit 1)
GoCompile: missing strict dependencies:
    /private/var/tmp/_bazel_kris/3ec06438aa9ef4cec825e8469e603db0/sandbox/darwin-sandbox/406/execroot/__main__/vendor/google.golang.org/grpc/internal/channelz/types_linux.go: import of "golang.org/x/sys/unix"
Known dependencies are:
    google.golang.org/grpc/connectivity
    google.golang.org/grpc/credentials
    google.golang.org/grpc/grpclog
Check that imports in Go sources match importpath attributes in deps.

[krishicks]

This is the BUILD file gazelle made for google.golang.org/grpc/internal/channelz:

load("@io_bazel_rules_go//go:def.bzl", "go_library")

go_library(
    name = "go_default_library",
    srcs = [
        "funcs.go",
        "types.go",
        "types_linux.go",
        "types_nonlinux.go",
        "util_linux_go19.go",
        "util_nonlinux_pre_go19.go",
    ],
    importmap = "vendor/google.golang.org/grpc/internal/channelz",
    importpath = "google.golang.org/grpc/internal/channelz",
    visibility = ["//vendor/google.golang.org/grpc:__subpackages__"],
    deps = [
        "//vendor/google.golang.org/grpc/connectivity:go_default_library",
        "//vendor/google.golang.org/grpc/credentials:go_default_library",
        "//vendor/google.golang.org/grpc/grpclog:go_default_library",
    ] + select({
        "@io_bazel_rules_go//go/platform:linux": [
            "//vendor/golang.org/x/sys/unix:go_default_library",
        ],
        "//conditions:default": [],
    }),
)

[krishicks]

It's also possible I'm just doing something stupid here. I'm using rules_docker's container_image to try and duplicate our Dockerfile-based images, and I wonder how bazel knows to compile a Linux binary for that container image. I'll keep investigating.

[krishicks]

My issue was fixed by specifying --platforms; is this the optimal way to go or is there an alternative?

bazel run --platforms=@io_bazel_rules_go//go/toolchain:linux_amd64

[krishicks]

To close this out, it seems --platforms is the way to go. From rules_go:

...it's almost always better to cross-compile by setting --platforms on the command line instead.

[jayconrod]

+1 to using --platforms. The goos and goarch attributes ignore select expressions, and we need them in several places to avoid trying to build dependencies that are unbuildable on certain platforms.

Bazel folks are working on exposing cross-platform transitions in the Starlark API. That should launch in the next few months. I'm hoping that will allow us to re-implement goos and goarch so they work the way you'd expect.

[skyl]

@krishicks I get much the same file from gazelle; but, I'm trying to run bazel test on darwin. So, using --platforms doesn't seem to be right and doesn't seem to work.

I'm vendoring everything with # gazelle:proto disable_global and using go mod.

So, my normal workflow is like:

cd to/my/gomod/file
if [ "$1" = "update" ]; then
    GO111MODULE=on go get -u
fi
GO111MODULE=on go mod vendor
GO111MODULE=on go mod tidy
bazel run //:gazelle
bazel test //...

I get the same GoCompile: missing strict dependencies: ... "golang.org/x/sys/unix"

My workaround for now is to modify the generated build files for, in my case:

• vendor/google.golang.org/grpc/internal/channelz/BUILD.bazel
• vendor/google.golang.org/grpc/internal/syscall/BUILD.bazel

If I manually make the deps like:

    deps = [
        "//go/src/vendor/google.golang.org/grpc/connectivity:go_default_library",
        "//go/src/vendor/google.golang.org/grpc/credentials:go_default_library",
        "//go/src/vendor/google.golang.org/grpc/grpclog:go_default_library",
    ] + select({
        "@io_bazel_rules_go//go/platform:linux": [
            "//go/src/vendor/golang.org/x/sys/unix:go_default_library",
        ],
        "//conditions:default": [
            **"//go/src/vendor/golang.org/x/sys/unix:go_default_library", # keep**
        ],
    }),

and

    deps = select({
        # ...
        "@io_bazel_rules_go//go/platform:darwin": [
            **"//go/src/vendor/golang.org/x/sys/unix:go_default_library", # keep **
            "//go/src/vendor/google.golang.org/grpc/grpclog:go_default_library",
        ],

then everything seems to work as expected. The problem then is, if I let go mod rewrite the vendor directory, I have to add back these lines to the dependencies. @jayconrod any suggestions for a workaround or maybe I should open a new issue?

[jayconrod]

@skyl Could you open a new issue? Please include a small repro and mention what version of rules_go, Bazel, and Gazelle you're using. I wasn't able to reproduce this on Linux or Darwin.

[kalbasit]

+1 to using --platforms. The goos and goarch attributes ignore select expressions, and we need them in several places to avoid trying to build dependencies that are unbuildable on certain platforms.

Bazel folks are working on exposing cross-platform transitions in the Starlark API. That should launch in the next few months. I'm hoping that will allow us to re-implement goos and goarch so they work the way you'd expect.

@jayconrod was this fixed upstream? I got a report today someone failing to build docker image (with rules_docker) of a go_binary specifying both goos and goarch.

[jayconrod]

Yes, it's fixed upstream. The rules_go work still needs to be done though.

[kalbasit]

Yes, it's fixed upstream. The rules_go work still needs to be done though.

Is there an issue we can follow for this fix?

[jayconrod]

I just filed #2219 to track this.