Go: cmd/go: duplicate symbols when including two main packages in -coverpkg in module mode

Created on 24 Feb 2019 · 25Comments · Source: golang/go

Note that I am not sure if this is a bug or expected behaviour, only that this does not happen in the latest stable 1.11 release and does on 1.12rc1.

What version of Go are you using (`go version`)?

$ go version
go version go1.12rc1 linux/amd64

Does this issue reproduce with the latest release?

Not the latest stable release (1.11.5)

What operating system and processor architecture are you using (`go env`)?

go env Output



$ go env

GOARCH="amd64"

GOBIN=""

GOCACHE="/home/lazer/.cache/go-build"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOOS="linux"

GOPATH="/home/lazer/go"

GOPROXY=""

GORACE=""

GOROOT="/home/lazer/sdk/go1.12rc1"

GOTMPDIR=""

GOTOOLDIR="/home/lazer/sdk/go1.12rc1/pkg/tool/linux_amd64"

GCCGO="gccgo"

CC="gcc"

CXX="g++"

CGO_ENABLED="1"

GOMOD="/home/lazer/Dev/gosnaillife/go.mod"

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build721704492=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I ran the following command:

go1.12rc1 test -race -v -coverpkg gitlab.com/drakonka/gosnaillife/cmd/snaillifecli,gitlab.com/drakonka/gosnaillife/cmd/snaillifesrv, -coverprofile allcoverage-rc.out ./...

Both of the above are main packages producing two binaries.

What did you expect to see?

I expected the tests to run and pass, with coverage output, as they do on Go 1.11.5

What did you see instead?

The following error:

2019/02/24 11:39:04 duplicate symbol runtime/debug.modinfo (types 30 and 30) in gitlab.com/drakonka/gosnaillife/cmd/snaillifecli and $WORK/b293/_pkg_.a(_go_.o)

I run Go RC and Go 1.11 side by side as part of my CI, which is how I noticed this issue. I am able to reproduce it locally as well. Full logs for the successful run of Go 1.11 and the failure run of Go 1.12rc1 can be found here:

FrozenDueToAge GoCommand NeedsFix

Source

lshulyay

👍3

All 25 comments

I found the same regression just a few days ago, but hadn't filed an issue yet. This is impeding my daily development, and I'd consider it a regression introduced in 1.12.

The 1.12 release will be out any day now though, so it's probably more realistic to target a backport to 1.12.1. /cc @bcmills @ianlancetaylor @hyangah since this very likely relates to the recent modinfo changes in the cycle.

mvdan on 25 Feb 2019

👍1

I'd consider it a regression introduced in 1.12.

This is almost certainly a bad interaction between #23910, //go:linkname, and the fix for #27584. Note that the former dates to at least Go 1.10, and can be a problem for any program that happens to use //go:linkname in package main (although those are hopefully rare).

Before the fix for #27584, the error (and redefinition) still existed, but because the variable was initialized in an init function, instead of a compile-time error one of the values of debug.modinfo would be overwritten at run-time.

bcmills on 25 Feb 2019

Given that we do not currently merge coverage profiles from multiple processes (#28235), I'm not sure why you would want to include two different main packages in the same -coverpkg argument (except, perhaps, to simplify the patterns passed to -coverpkg).

The workaround for now is to include at most one main package in the -coverpkg arguments, or to run coverage in GOPATH mode.

bcmills on 25 Feb 2019

👍1

CC @jayconrod

bcmills on 25 Feb 2019

Seems very similar to the issues I had when Golang 1.11 came out #23910. I guess I'm just disabling -coverpkg on CI for now.

schoentoon on 27 Feb 2019

Seems very similar to the issues I had when Golang 1.11 came out #23910. I guess I'm just disabling -coverpkg on CI for now.

Depending on the details of your setup that seems like it might be overkill when you can just remove the conflicting main packages from -coverpkg and still get coverage reporting, right?

lshulyay on 27 Feb 2019

Same issue on my end https://travis-ci.org/markus-wa/demoinfocs-golang/jobs/499992529

I see the milestone is 1.13, does that mean it won't be fixed in 1.12.x?

markus-wa on 28 Feb 2019

@bcmills I didn't know this was silently broken in 1.11; thanks for pointing this out. I think temporarily working around the issue for now is acceptable, but perhaps we should provide a simple example or one-liner that people could switch to.

mvdan on 28 Feb 2019

@mvdan , the one-liner is GO111MODULE=off. (There is a reason I wasn't comfortable turning it on-by-default in 1.12, and bugs like this one are that reason: they're unusual but not esoteric.)

bcmills on 28 Feb 2019

Thanks for the tip! If the bug(s) can be fixed in time for 1.13, then the timing with GO111MODULE=on would be ideal :)

mvdan on 28 Feb 2019

the one-liner is GO111MODULE=off

@bcmills but with that set to off, go, understandably, isn't able to find dependencies. - .travis.yml

Am I missing something obvious here?

I'm not sure why you would want to include two different main packages in the same -coverpkg argument

In my case I have a couple of examples for my lib and each of them has it's own main, then I just do go test -coverpkg=./... ./...

markus-wa on 1 Mar 2019

Change https://golang.org/cl/164877 mentions this issue: cmd/go: avoid link errors when -coverpkg covers main packages

gopherbot on 1 Mar 2019

🎉1

I was bitten by the same problem: multiple main packages in subdirectories below examples/.

If you're slightly insane, here's a work around:

go test -race \
    -coverpkg .,$(
        ls -1d */ \
        | sed -E \
            -e :a \
            -e '$!N ; s|\n|,| ; ta' \
            -e 's#dist/|docs/|examples/|scripts/##g' \
            -e 's/,,/,/g ; s/^,// ; s/,$//' \
            -e 's|/|/...|g'
    ) \
    -coverprofile=dist/coverage.txt \
    -tags integration ./...

sjansen on 1 Mar 2019

@sjansen You can also use go list instead, if that's more sane.

go test -v -coverprofile=coverage.out -coverpkg=$(go list ./... | grep -v 'examples/' | awk -vORS=, '{ print $1 }' | sed 's/,$/\n/')

I'm still a bit bummed to lose coverage on the examples though 😄.

markus-wa on 2 Mar 2019

I've been doing it like this:

CVPKG=$(go list ./... | grep -v -e mocks -e snaillifecli -e snaillifesrv | tr '\n' ',') 

go test -race -v -coverpkg $CVPKG -coverprofile allcoverage-stable.out ./...

lshulyay on 2 Mar 2019

👍1

@gopherbot, please backport to 1.12. This is a significant regression from GOPATH mode and the fix is straightforward.

bcmills on 8 Mar 2019

👍1

Backport issue(s) opened: #30684 (for 1.12).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

gopherbot on 8 Mar 2019

I can still reproduce this in 1.12.1:

> docker run -it golang:1.12.1
# cd ..
# git clone https://github.com/google/go-cloud
# cd go-cloud
# export GO111MODULE=on
# go test -coverpkg=./... ./...
...
2019/03/18 12:52:58 duplicate symbol runtime/debug.modinfo (types 30 and 30) in gocloud.dev/samples/blobdownload and $WORK/b466/_pkg_.a(_go_.o)

(Note: I had to run the go test line several times because I would get occasional "Could not resolve host: github.com" errors. Not sure whether that's specific to my setup.)

jba on 18 Mar 2019

Reproduced. Trying to narrow it down.

It's possible this is a new issue. The _go_.o part of the message is suspicious. I don't think that file should be getting linked.

jayconrod on 18 Mar 2019

It might be useful to know that this repo has nested modules.

jba on 18 Mar 2019

Change https://golang.org/cl/168200 mentions this issue: cmd/go: avoid link error when -coverpkg covers main packages (more)

gopherbot on 18 Mar 2019

@gopherbot Please backport to 1.12. This is a more complete fix for an issue that was supposed to be fixed in 1.12.1.

jayconrod on 19 Mar 2019

For posterity, the second backport issue was created manually: #30937 (for 1.12).

dmitshur on 21 Mar 2019

I am encountering errors on go1.12.5 just like previously reported.

$ go version
go version go1.12.5 linux/amd64

go env Output



$ go env

GOARCH="amd64"

GOBIN=""

GOCACHE="/home/simon/.cache/go-build"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOOS="linux"

GOPATH="/media/ext4_data/Coding/go"

GOPROXY=""

GORACE=""

GOROOT="/media/ext4_data/Linux/source/go"

GOTMPDIR=""

GOTOOLDIR="/media/ext4_data/Linux/source/go/pkg/tool/linux_amd64"

GCCGO="gccgo"

CC="gcc"

CXX="g++"

CGO_ENABLED="1"

GOMOD="/home/simon/tmp/syncthing/go.mod"

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build248314634=/tmp/go-build -gno-record-gcc-switches"

Running all tests with -coverpkg on all packages

git clone https://github.com/syncthing/syncthing.git
cd syncthing
go test -short -coverprofile coverage.out -coverpkg ./cmd/...,./lib/... ./cmd/... ./lib/...

results in errors like previously reported:

2019/05/15 16:02:50 duplicate symbol go.constinfo.main (types 45 and 45) in github.com/syncthing/syncthing/cmd/strelaypoolsrv and /home/simon/.cache/go-build/87/87afda35dba0b8de44e2f9d449eda7ce4caef7ed781612883b4e457eab1ee3d5-d(_go_.o)

The same also happens with just ./.... I cleaned out ~/.cache/go-build for good measure, didn't change anything.

To make sure it isn't some weird local problem of mine I also ran it on Syncthing's CI and reproduced the issue there as well: https://build.syncthing.net/viewLog.html?buildId=39242&buildTypeId=Syncthing_Coverage&tab=buildLog