Go: cmd/go: test coverpkg=all ./... with test only packages will fail to build

[AlexRouSg]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.11

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

linux amd64

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

Ran go test -coverpkg=all ./... with the following folder structure:

├── foo.go
├── foo_test.go
└── tests
    └── tests_test.go

Content of files is irrelevant.
Apparently go test -cover ./... works, so it's specific to -coverpkg
Also deleting foo_test.go works

What did you expect to see?

ok      _/<HOME>/go/src/foo   0.001s  coverage: 0.0% of statements [no tests to run]
ok      _/<HOME>/go/src/foo/tests     0.004s  coverage: 0.0% of statements [no tests to run]

What did you see instead?

go build _/<HOME>/go/src/foo/tests: no non-test Go files in go/src/foo/tests
FAIL    _/<HOME>/go/src/foo [build failed]
ok      _/<HOME>/go/src/foo/tests     0.001s  coverage: 0.0% of statements in all [no tests to run]

[FiloSottile]

/cc @bcmills @rsc

[vvakame]

I met this issue in real world.
This issue is very painful...
because, unrelated packages will be build failed without an error message...
I found go build go.mercari.io/datastore/testbed: no non-test Go files in /go/src/go.mercari.io/datastore/testbed message, but I can't found relation between this message to build faled. and I see it occasionally, but I never had a problem until now.

https://github.com/mercari/datastore/tree/v1.4.0/testbed
https://circleci.com/gh/mercari/datastore/219

$ go test go.mercari.io/datastore go.mercari.io/datastore/aedatastore go.mercari.io/datastore/aeprodtest go.mercari.io/datastore/boom go.mercari.io/datastore/clouddatastore go.mercari.io/datastore/dsmiddleware go.mercari.io/datastore/dsmiddleware/aememcache go.mercari.io/datastore/dsmiddleware/chaosrpc go.mercari.io/datastore/dsmiddleware/dslog go.mercari.io/datastore/dsmiddleware/fishbone go.mercari.io/datastore/dsmiddleware/localcache go.mercari.io/datastore/dsmiddleware/noop go.mercari.io/datastore/dsmiddleware/rediscache go.mercari.io/datastore/dsmiddleware/rpcretry go.mercari.io/datastore/dsmiddleware/storagecache go.mercari.io/datastore/internal go.mercari.io/datastore/internal/shared go.mercari.io/datastore/internal/testutils go.mercari.io/datastore/testbed go.mercari.io/datastore/testsuite go.mercari.io/datastore/testsuite/dsmiddleware/dslog go.mercari.io/datastore/testsuite/dsmiddleware/fishbone go.mercari.io/datastore/testsuite/dsmiddleware/localcache go.mercari.io/datastore/testsuite/dsmiddleware/rpcretry go.mercari.io/datastore/testsuite/favcliptools go.mercari.io/datastore/testsuite/realworld/recursivebatch go.mercari.io/datastore/testsuite/realworld/tbf -count 1 -p 1 -coverpkg=go.mercari.io/datastore/... -covermode=atomic -coverprofile=coverage.txt
go build go.mercari.io/datastore/testbed: no non-test Go files in /go/src/go.mercari.io/datastore/testbed
FAIL    go.mercari.io/datastore [build failed]
FAIL    go.mercari.io/datastore/aedatastore [build failed]
?       go.mercari.io/datastore/aeprodtest  [no test files]
FAIL    go.mercari.io/datastore/boom [build failed]
FAIL    go.mercari.io/datastore/clouddatastore [build failed]
?       go.mercari.io/datastore/dsmiddleware    [no test files]
FAIL    go.mercari.io/datastore/dsmiddleware/aememcache [build failed]
FAIL    go.mercari.io/datastore/dsmiddleware/chaosrpc [build failed]
FAIL    go.mercari.io/datastore/dsmiddleware/dslog [build failed]
FAIL    go.mercari.io/datastore/dsmiddleware/fishbone [build failed]
FAIL    go.mercari.io/datastore/dsmiddleware/localcache [build failed]
?       go.mercari.io/datastore/dsmiddleware/noop   [no test files]
FAIL    go.mercari.io/datastore/dsmiddleware/rediscache [build failed]
FAIL    go.mercari.io/datastore/dsmiddleware/rpcretry [build failed]
FAIL    go.mercari.io/datastore/dsmiddleware/storagecache [build failed]
?       go.mercari.io/datastore/internal    [no test files]
?       go.mercari.io/datastore/internal/shared [no test files]
?       go.mercari.io/datastore/internal/testutils  [no test files]
ok      go.mercari.io/datastore/testbed 6.887s  coverage: 0.7% of statements in go.mercari.io/datastore/...
?       go.mercari.io/datastore/testsuite   [no test files]
?       go.mercari.io/datastore/testsuite/dsmiddleware/dslog    [no test files]
?       go.mercari.io/datastore/testsuite/dsmiddleware/fishbone [no test files]
?       go.mercari.io/datastore/testsuite/dsmiddleware/localcache   [no test files]
?       go.mercari.io/datastore/testsuite/dsmiddleware/rpcretry [no test files]
?       go.mercari.io/datastore/testsuite/favcliptools  [no test files]
?       go.mercari.io/datastore/testsuite/realworld/recursivebatch  [no test files]
?       go.mercari.io/datastore/testsuite/realworld/tbf [no test files]

[gopherbot]

Change https://golang.org/cl/164198 mentions this issue: cmd/go: ignore test only packages with -coverpkg

[BinaryHexer]

This is something that has just bit me as well.

Anyways if anyone is stuck because of this like me, here is tiny workaround:

go test -v -race -coverpkg=$(go list ./... grep -v "/test") -coverprofile=coverage.txt $(go list ./...)

The above workaround only works when your directory structure is as follows:

├── foo.go
└── test
    └── foo_test.go
├── bar.go
└── test
    └── bar_test.go

That is your test only package are always inside a directory named test.
If you have different naming style please modify the grep -v "/test" to suit your use case.

[byxor]

I found a tool called go-acc that solved this problem for me. :tada:

# Download and install go-acc
$ go get -u github.com/ory/go-acc

# Run it against some package(s)
$ go-acc github.com/some/package
$ go-acc .
$ go-acc ./...
$ go-acc $(glide novendor)

Here's the blogpost I found this tool on: https://www.ory.sh/golang-go-code-coverage-accurate

Thanks ory; if you see this, you saved me hours of pain.

[renld]

when i use go1.14, it appears too..

What version of Go are you using (go version)?

$ go version
go version go1.14.1 linux/amd64

Does this issue reproduce with the latest release?

not try...

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOARCH="amd64"

What did you do?

example
/

• main.go
• main_test.go
• pkg1/a_test.go
• pkg2/b.go
• pkg2/b_test.go

go test -gcflags=-l -test.coverpkg=./... ./pkg1 => test build ok
go test -gcflags=-l ./... => test build ok
go test -gcflags=-l -test.coverpkg=./... ./... => test build fail when some sub dir only contains _test.go

after rm pkg1/a_test.go
/

• main.go
• main_test.go
• pkg2/b.go
• pkg2/b_test.go
  go test -gcflags=-l -test.coverpkg=./... ./... => test build ok

What did you expect to see?

go test -gcflags=-l -test.coverpkg=./... ./... test build ok

What did you see instead?

go test -gcflags=-l -test.coverpkg=./... ./... [build failed] ( include pkg2)

[hitzhangjie]

go test -v -race -coverpkg=$(go list ./... grep -v "/test") -coverprofile=coverage.txt $(go list ./...)

I just come across this problem as well.

I tested with GOTMPDIR=. go test ...... , then 2 tmpdir is generated under current directory. I inspect the directory content and find that:
for each package something like 'hello', it will generate 3 files:

• _testmain.go
• importcfg.link
• a built binary hello.test

I think the hello.test is built upon _testmain.go.

In _testmain.go, there's some function calls like coverRegisterFile(path/to/some.go).

Apparently, it registers some duplicate go files that doesn't import in current package 'hello'.

If coverRegisterFile(path/go/some.go is a file represents *.pb.go, go test -cover panics.

Maybe the flags passed to go test caused the duplicate coverRegisterFile statements.

I use this for i ingo list ./...; do go test -v -coverpkg=$i -covermode=count -coverprofile=cover.out 2>&1 | tee -a report.out; done to work around this problem.

[itsmeadi]

The coverpkg expects comma seperated package names, so the workaround that worked for me
go test ./... -coverpkg=$(go list ./... | grep -v test | tr "\n" ",") -coverprofile=coverage.out assuming your test package name contains test as a substring

[rspechenkin]

Hello, any chance that this will be fixed anytime soon? This issue if very painful for us