Go: cmd/go: go mod tidy adds many indirect modules

[dvrkps]

Please answer these questions before submitting your issue. Thanks!

What did you do?

go mod tidy

What did you expect to see?

cat go.mod

module github.com/dvrkps/consuldojo

require (
    github.com/hashicorp/consul v1.2.2
    github.com/hashicorp/go-cleanhttp v0.5.0 // indirect
    github.com/hashicorp/go-rootcerts v0.0.0-20160503143440-6bb64b370b90 // indirect
    github.com/hashicorp/serf v0.8.1 // indirect
    github.com/mitchellh/go-homedir v1.0.0 // indirect
    github.com/mitchellh/mapstructure v1.0.0 // indirect
)

What did you see instead?

go mod tidy
go: finding github.com/pascaldekloe/goe/verify latest
go: finding github.com/hashicorp/go-msgpack/codec latest
go: finding github.com/pascaldekloe/goe latest
go: finding github.com/hashicorp/go-msgpack latest
go: finding github.com/armon/go-metrics latest
go: finding github.com/stretchr/testify/require latest
go: finding github.com/hashicorp/go-sockaddr latest
go: finding github.com/pmezard/go-difflib/difflib latest
go: finding github.com/davecgh/go-spew/spew latest
go: finding github.com/sean-/seed latest
go: finding github.com/DataDog/datadog-go/statsd latest
go: finding github.com/hashicorp/yamux latest
go: finding github.com/DataDog/datadog-go latest
go: finding github.com/prometheus/client_golang/prometheus latest
go: finding gopkg.in/check.v1 latest
go: finding google.golang.org/appengine/datastore latest
go: finding golang.org/x/net/ipv4 latest
go: finding golang.org/x/net/ipv6 latest
go: finding golang.org/x/net latest
go: finding golang.org/x/crypto/ed25519 latest
go: finding golang.org/x/crypto latest
go: finding github.com/beorn7/perks/quantile latest
go: finding github.com/prometheus/client_model/go latest
go: finding github.com/circonus-labs/circonusllhist latest
go: finding github.com/hashicorp/go-retryablehttp latest
go: finding github.com/prometheus/common/expfmt latest
go: finding github.com/prometheus/common/model latest
go: finding github.com/tv42/httpunix latest
go: finding github.com/beorn7/perks latest
go: finding github.com/prometheus/client_model latest
go: finding github.com/prometheus/common latest
go: finding github.com/prometheus/procfs latest
go: finding github.com/golang/protobuf/proto latest
go: finding github.com/matttproud/golang_protobuf_extensions/pbutil latest
go: finding golang.org/x/sync/errgroup latest
go: finding golang.org/x/sync latest

and go.mod looks like:

module github.com/dvrkps/consuldojo

require (
    github.com/DataDog/datadog-go v0.0.0-20180822151419-281ae9f2d895 // indirect
    github.com/armon/go-metrics v0.0.0-20180713145231-3c58d8115a78 // indirect
    github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973 // indirect
    github.com/circonus-labs/circonus-gometrics v2.2.1+incompatible // indirect
    github.com/circonus-labs/circonusllhist v0.0.0-20180430145027-5eb751da55c6 // indirect
    github.com/davecgh/go-spew v1.1.1 // indirect
    github.com/golang/protobuf v1.2.0 // indirect
    github.com/hashicorp/consul v1.2.2
    github.com/hashicorp/go-cleanhttp v0.5.0 // indirect
    github.com/hashicorp/go-immutable-radix v1.0.0 // indirect
    github.com/hashicorp/go-msgpack v0.0.0-20150518234257-fa3f63826f7c // indirect
    github.com/hashicorp/go-multierror v1.0.0 // indirect
    github.com/hashicorp/go-retryablehttp v0.0.0-20180718195005-e651d75abec6 // indirect
    github.com/hashicorp/go-rootcerts v0.0.0-20160503143440-6bb64b370b90 // indirect
    github.com/hashicorp/go-sockaddr v0.0.0-20180320115054-6d291a969b86 // indirect
    github.com/hashicorp/memberlist v0.1.0 // indirect
    github.com/hashicorp/serf v0.8.1 // indirect
    github.com/hashicorp/yamux v0.0.0-20180826203732-cc6d2ea263b2 // indirect
    github.com/kr/pretty v0.1.0 // indirect
    github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
    github.com/miekg/dns v1.0.8 // indirect
    github.com/mitchellh/go-homedir v1.0.0 // indirect
    github.com/mitchellh/go-testing-interface v1.0.0 // indirect
    github.com/mitchellh/mapstructure v1.0.0 // indirect
    github.com/pascaldekloe/goe v0.0.0-20180627143212-57f6aae5913c // indirect
    github.com/pkg/errors v0.8.0 // indirect
    github.com/pmezard/go-difflib v1.0.0 // indirect
    github.com/prometheus/client_golang v0.8.0 // indirect
    github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910 // indirect
    github.com/prometheus/common v0.0.0-20180801064454-c7de2306084e // indirect
    github.com/prometheus/procfs v0.0.0-20180725123919-05ee40e3a273 // indirect
    github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 // indirect
    github.com/stretchr/testify v1.2.2 // indirect
    github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926 // indirect
    golang.org/x/crypto v0.0.0-20180830192347-182538f80094 // indirect
    golang.org/x/net v0.0.0-20180826012351-8a410e7b638d // indirect
    golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f // indirect
    google.golang.org/appengine v1.1.0 // indirect
    gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 // indirect
    gopkg.in/vmihailenco/msgpack.v2 v2.9.1 // indirect
)

System details

go version go1.11 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/davor/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/davor"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/davor/src/github.com/dvrkps/consuldojo/go.mod"
GOROOT/bin/go version: go version go1.11 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.11
uname -v: Darwin Kernel Version 17.7.0: Thu Jun 21 22:53:14 PDT 2018; root:xnu-4570.71.2~1/RELEASE_X86_64
ProductName:    Mac OS X
ProductVersion: 10.13.6
BuildVersion:   17G65
lldb --version: lldb-902.0.79.7
  Swift-4.1

[thepudds]

@dvrkps a few quick comments, including I will attempt to share my personal understanding.

go mod tidy ensures your current go.mod reflects all possible build tags/OS/architecture combinations (as described here).

In contrast, other commands like go build and go test only update go.mod based on the current build invocation's tags/OS/architecture.

go mod tidy can also pull in test dependencies that go build might not.

More generally, part of the reason modules exist is to record precise dependency requirements (in go.mod) and create reproducible builds. If a particular dependency does not itself have a go.mod (because the dependency has not yet opted in to modules itself), then the dependency lacking a go.mod will have _its_ dependencies recorded in a parent go.mod (e.g., your go.mod), along with an // indirect comment to indicate that the recorded information is from an indirect dependency rather than a direct dependency.

In light of that, could you comment a little more on what aspect of your results is unexpected here?

Also, if you are curious why a particular module is showing up in your go.mod, then running go mod why -m <module> is one way to answer that question:
https://tip.golang.org/cmd/go/#hdr-Explain_why_packages_or_modules_are_needed

There are other useful tools for inspecting requirements and versions, such as go mod graph and go list -m all.

Finally, I am a member of the community just like you, so happy to learn if anything I wrote above is incorrect.

[dvrkps]

@thepudds thx for looking.

My repo is just playground. I use only Consul as dependency.

I’m not sure that all indirect modules are consul dependencies, but they seem to be.

Prometheus, Datadog, Appengine, ...

[dvrkps]

Nothing strange here anymore. I close it. Thx.