Go: cmd/go: 'go build' in module mode rebuilds vendored dependencies in GOROOT

[vincentbernat]

What version of Go are you using (go version)?

go version go1.11 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bernat/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/bernat/src/gocode"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/bernat/tmp/reproduce/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build345235277=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Using the following main.go file:

package main

import (
    "fmt"
    "net/http"
)

func main() {
    helloHandler := func(w http.ResponseWriter, req *http.Request) {
        io.WriteString(w, "Hello, world!\n")
    }

    http.HandleFunc("/hello", helloHandler)
    http.ListenAndServe(":8080", nil)
}

And the following commands:

go mod init anything
go build -i -o /dev/null

I get:

go: finding golang.org/x/text v0.3.0
go build golang_org/x/crypto/cryptobyte/asn1: open /usr/lib/go-1.11/pkg/linux_amd64/vendor/golang_org/x/crypto/cryptobyte/asn1.a: permission denied
go build golang_org/x/crypto/poly1305: open /usr/lib/go-1.11/pkg/linux_amd64/vendor/golang_org/x/crypto/poly1305.a: permission denied
go build golang_org/x/crypto/internal/chacha20: open /usr/lib/go-1.11/pkg/linux_amd64/vendor/golang_org/x/crypto/internal/chacha20.a: permission denied
go build golang_org/x/net/dns/dnsmessage: open /usr/lib/go-1.11/pkg/linux_amd64/vendor/golang_org/x/net/dns/dnsmessage.a: permission denied
go build golang_org/x/crypto/curve25519: open /usr/lib/go-1.11/pkg/linux_amd64/vendor/golang_org/x/crypto/curve25519.a: permission denied
go build golang_org/x/text/transform: open /usr/lib/go-1.11/pkg/linux_amd64/vendor/golang_org/x/text/transform.a: permission denied
go build golang_org/x/text/unicode/bidi: open /usr/lib/go-1.11/pkg/linux_amd64/vendor/golang_org/x/text/unicode/bidi.a: permission denied
go build golang_org/x/net/http2/hpack: open /usr/lib/go-1.11/pkg/linux_amd64/vendor/golang_org/x/net/http2/hpack.a: permission denied

What did you expect to see?

go build shouldn't try to erase stuff shipped with Go. This only seems to happen for vendorized stuff. Non-vendorized stuff don't get compiled. Without -i, this doesn't happen. The provided command is the one issued by a flycheck plugin for Emacs.

[bradfitz]

I think this is a duplicate of #26988.

[ianlancetaylor]

I thought #26998 was fixed by https://golang.org/cl/130138, although I see that the CL does not mention the issue.

This should be fixed, but, in the meantime, don't use go build -i. Now that we have the build cache, go build -i isn't useful.

[FiloSottile]

@gopherbot please file this for backport against 1.11. This is a regression.

[gopherbot]

Backport issue(s) opened: #27389 (for 1.11).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

[ianlancetaylor]

Honestly I'm not sure it helps to file an issue for backport when we don't have a fix yet.

[FiloSottile]

@ianlancetaylor the point of opening a backport issue is that it will not get closed when a "Fixes" patch gets merged. We used to lose track of things that way. Opening it immediately avoids having to track an extra "should backport but don't have a fix yet" state separately, and having to take a GitHub action when it transitions to "it has a fix now" by a Gerrit action.

releasebot pushes all CherryPickCandidate issues to the next minor release, so we don't even have to manage them, making them basically zero overhead.

[ianlancetaylor]

But the downside is that when we are looking at a minor release, we have to look at these issues again and again, each time going back to the original issue, and working out that it has not been fixed and there is nothing to backport.

Since it is gopherbot that is closing the issues, we don't have to be hostage to it. I suggest that if gopherbot sees that the issue has a milestone of an earlier minor release, that the issue not be closed when the patch is committed to tip.

[FiloSottile]

@ianlancetaylor It's GitHub, not gopherbot, that closes issues when a patch is submitted to master. (gopherbot only takes care of merges to release branches.) The other issue is when something needs backport to two releases, which we also used to lose track of. It sounds like a gopherbot feature that only applies the CherryPickCandidate label when the main issue is closed would solve this? In any case, we are OT, so if you'd like to propose that or a different change, please open a separate issue.

[FiloSottile]

More details about this rebuilding issue at #27482.

[godwhoa]

For anyone running into this issue, try the current master. Fixed in my case.

➜  bug27285 go version
go version go1.11rc1 linux/amd64
➜  bug27285 go build -i
go build golang_org/x/crypto/poly1305: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/crypto/poly1305.a: permission denied
go build golang_org/x/crypto/curve25519: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/crypto/curve25519.a: permission denied
go build golang_org/x/text/transform: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/text/transform.a: permission denied
go build golang_org/x/text/unicode/bidi: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/text/unicode/bidi.a: permission denied
go build golang_org/x/net/http2/hpack: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/net/http2/hpack.a: permission denied
go build golang_org/x/net/dns/dnsmessage: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/net/dns/dnsmessage.a: permission denied
go build golang_org/x/crypto/cryptobyte/asn1: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/crypto/cryptobyte/asn1.a: permission denied
go build golang_org/x/crypto/internal/chacha20: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/crypto/internal/chacha20.a: permission denied

➜  bug27285 go version
go version devel +ceb7745 Fri Sep 7 18:58:57 2018 +0000 linux/amd64
➜  bug27285 go build -i
# anything
./main.go:4:2: imported and not used: "fmt"
./main.go:10:3: undefined: io

[bcmills]

@godwhoa, I see traces there for rc1 and devel. Have you tried the actual 1.11 release? (I believe some module bugs were fixed in between rc1 and rc2.)

[godwhoa]

➜  bug27285 go version
go version go1.11 linux/amd64
➜  bug27285 go build -i
go build golang_org/x/crypto/cryptobyte/asn1: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/crypto/cryptobyte/asn1.a: permission denied
go build golang_org/x/net/dns/dnsmessage: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/net/dns/dnsmessage.a: permission denied
go build golang_org/x/crypto/curve25519: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/crypto/curve25519.a: permission denied
go build golang_org/x/crypto/internal/chacha20: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/crypto/internal/chacha20.a: permission denied
go build golang_org/x/crypto/poly1305: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/crypto/poly1305.a: permission denied
go build golang_org/x/text/transform: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/text/transform.a: permission denied
go build golang_org/x/net/http2/hpack: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/net/http2/hpack.a: permission denied
go build golang_org/x/text/unicode/bidi: open /usr/local/go/pkg/linux_amd64/vendor/golang_org/x/text/unicode/bidi.a: permission denied

[bcmills]

Thanks!

[bcmills]

My first clue so far is that the effective ImportPath of the vendored dependencies varies between GOPATH mode and module mode: in GOPATH mode they have the prefix vendor/, where in module mode they do not.

I suspect that that results in different artifacts and/or different cache keys.

[bcmills]

(And I have a test case that replicates this.)

[bcmills]

Here we go:
https://github.com/golang/go/blob/77e503a3224ada21cc84ab9078980a7d4230492a/src/cmd/go/internal/work/exec.go#L180

[gopherbot]

Change https://golang.org/cl/136138 mentions this issue: cmd/go: retain vendor/ prefix on standard-library imports during loading

[rsc]

I am not convinced this is needed in a point release. The fix is to stop using go build -i. That was a workaround for the lack of build caching, but build caching was introduced in Go 1.10. There are _no_ supported go releases where -i is necessary for build performance.

[rsc]

In module mode I already thought it was the case that we literally never wrote to the pkg directory. We should not write to the package directory in this case either. That's the bug.

[davecheney]

@rsc if it is no longer of use, should the -i flag be removed from go build?

[bcmills]

We'll make it a no-op in module mode.

[Foxboron]

Is this issue possibly connected with #24034?

[bcmills]

Possibly. The fix in both cases may be to make install a no-op for files in GOROOT.

[gopherbot]

Change https://golang.org/cl/147443 mentions this issue: vendor/golang_org: move to internal/golang_org