Rules_go: Integration with Go editor tooling

[nelhage-stripe]

I'm not sure if this is a feature request or just a request for thoughts, but are there any recommended techniques for making rules_go-built repositories play nice with Go editor tooling like gocode and godefs and goimports? I check out my repository into the place it would go on GOPATH, which works well for many purposes, but if I generate code using bazel (e.g. to compile protobufs), goimports and godef can't see it. goimports even sometimes gets confused and tries to insert a .../bazel-bin/... import.

Are there any thoughts or best practices about how to make this experience better? Do we need to write bazel-aware versions of all these tools? Should we perhaps add a rule to create a symlink-farm pseudo-GOPATH inside bazel-bin we can point editor tooling at?

[alandonovan]

My team is actively working to solve the problem of how to build source code tools that work unmodified in any workspace---go build, bazel, or blaze. We hope to share some prototype tools, an API, and a proposal document later this quarter.

cc: @matloob

[kgreenek]

Major +1 for this.
In addition, IDE's don't know how to handle .go files generated from protobufs. Our current workaround is to run a script that copies the generated *.pb.go files from the gen dir to the proto directory in source.

[kerinin]

:+1: for a "preferred" method of exposing generated code to the standard toolchain. It would be nice if similar code-generation tools like gomock could be used.

[toffaletti]

A command or some manual steps to get a GOPATH would be good enough for now. Not having anything is really painful right now.

[nelhage-stripe]

I briefly experimented with a proof-of-concept aspect that can generate a GOPATH symlink farm from a rules_go target, but I abandoned it because most of the tools I looked at didn't like following symlinks, so it didn't help that much. It might be an approach worth pursing, though. I can post the code somewhere if someone'd find it useful.

[micanzhang]

is it possible write a command tool which generate go source files and copy these go files into GOPATH? or copy to ./vendor directory?

[jayconrod]

There is an experimental rule that does this in path.bzl. You can use this as:

load("@io_bazel_rules_go//go:def.bzl", "go_path")

go_path(
    name = "path",
    deps = [":go_default_library"],
)

Building this rule will produce a GOPATH-like directory for the deps (which may be libraries, binaries, and tests) and all their transitive dependencies.

[codesuki]

@alandonovan Is there any progress on this?
My team is actively working to solve the problem of how to build source code tools that work unmodified in any workspace---go build, bazel, or blaze. We hope to share some prototype tools, an API, and a proposal document later this quarter.

[ianthehat]

There is progress, but nothing to share yet. It has finally risen to the top of the priority pile though, and being implemented now.

[codesuki]

Glad to hear that!

[iamneal]

Building this rule will produce a GOPATH-like directory for the deps (which may be libraries, binaries, and tests) and all their transitive dependencies.

@jayconrod when you say you can rely on binaries and tests do you mean like this?

go_path(
    name = "path",
    deps = [
        ":go_default_library",
        ":go_default_xtest",
    ],
)

this fails for me.

It would be nice to have this work though, because go_default_xtest has some dependencies that are not pulled in from go_default_library.

[jayconrod]

@iamneal I think depending on a test is intended to work, but looking at path.bzl, that appears to be broken.

Could you file a new issue with a little more information about your setup and the errors you're getting?

[iamneal]

@jayconrod I filed an issue right here https://github.com/bazelbuild/rules_go/issues/1329 If there is any other info that you need let me know.

[jayconrod]

@iamneal Thanks that's exactly what I need.

[Goodwine]

Are there any updates that can be shared?

[jayconrod]

I'm working on a small program that produces metadata for Bazel workspaces. That will be consumed by golang.org/x/tools/go/packages. Several members of the Go tools team are pushing out updates to commonly used tools to migrate to golang.org/x/tools/go/packages. Once this migration work is complete, tools will be able to query golang.org/x/tools/go/packages for workspace metadata and type information, and they'll get an answer for Bazel workspaces without needing to integrate any Bazel-specific code.

[Globegitter]

@jayconrod Is there any update on the program to be consumed by packages?

[jayconrod]

Sorry, no update yet. I'm hoping to get it done by the end of the year.

[prestonvanloon]

Any update on this? I'm hoping to run gometalinter with the bazel managed dependencies.

[shettich]

Any news? It seems like it's fixed and my just need an entry in the FAQ or something?

[jayconrod]

@shettich It's not fixed. #512 is the main issue for writing a driver for golang.org/x/tools/go/packages, which is needed to make editors work.

These two problems are distinct, so I'm not closing either as a duplicate, but I'd prefer to keep discussion on #512.

[naturalwarren]

GoLand by Intellij just works with the Bazel plugin. The Bazel-out gets symlinked into libraries.

[jayconrod]

@naturalwarren GoLand has their own proprietary implementation of all this. It doesn't help people using other editors and can't be used to make other tools work with Bazel.

[bobg]

My preferred approach is to keep the generated code in the source repo, so all tooling Just Works; but to verify at bazel build time that it's up to date with respect to its sources. I don't yet know Bazel rule-writing well enough to know how to approach this, however.

(I'm aware that some will consider this counter to the Bazel philosophy of no-build-products-in-the-source-tree, but I think that's a misguided antipattern. Source code - even generated source code - has uses other than just being turned into object code, after all, including being something people read to understand how a system works.)