go 1.11.4 mod checksum errors

[nictuku]

$ go version
go version go1.11.4 linux/amd64

$ go get ./...

or

$ go mod download
go: verifying gotest.[email protected]+incompatible: checksum mismatch
downloaded: h1:VsBPFP1AI068pPrMxtb/S8Zkgf9xEmTLJjfM+P5UIEo=
go.sum: h1:y0IMTfclpMdsdIbr6uwmJn5/WZ7vFuObxDMdrylFM3A=

Switch to Go 1.11.3

Nuke $GOPATH/pkg/mod/cache

Repeat go get ./... or go mod download, and it works.

I'm creating a simple repo with repro steps now.

[nictuku]

Perhaps this was because the checksums created by 1.11.3 were the "wrong" ones? So nuking the cache is probably fine?

[dotwaffle]

No, definitely not 1.11.3, my checksum was made back in August and is broken as of 1.11.4, whereas it worked with 1.11 thru 1.11.3 without showing errors:

go: verifying github.com/grpc-ecosystem/[email protected]: checksum mismatch
    downloaded: h1:Iju5GlWwrvL6UBg4zJJt3btmonfrMlCDdsejg4CZE7c=
    go.sum:     h1:BWIsLfhgKhV5g/oF34aRjniBHLTZe5DNekSjbAjIS6c=

[agnivade]

/cc @bcmills @myitcv

[bcmills]

Yes, pre-1.11.4, modules containing symlinks were constructed erroneously. 1.11.4 correctly omits those symlinks, which may change the sums for a few (hopefully isolated) modules.

[bcmills]

See #27093.

[nictuku]

It sounds like we can close this issue. It's an inconvenient but ultimately benign error caused by _previous_ versions.

It can be worked around by manually updating (or simply removing) the appropriate lines from go.sum.

[thepudds]

Note: if you have a copy of a module with symlinks in your module cache, you likely will need to run go clean -modcache in order to get 1.11.4 to generate the new (corrected) checksum. (e.g., see #29282)

[VojtechVitek]

With Go's backward compatibility rule in mind, I would expect that Go 1.11.4+ would handle the old "wrong" symlink checksums without having to error out. Was this considered at all?

We've hit this issue three times already on a single codebase, just because of the module cache that each developer has on their machines. We didn't know that we all have to perform go clean -modcache after upgrading to Go 1.11.4.

[bcmills]

With Go's backward compatibility rule in mind, I would expect that Go 1.11.4+ would handle the old "wrong" symlink checksums without having to error out. Was this considered at all?

Given a module zip file without symlink files, there is no way to compute whether that module corresponds to a given hash with the redacted files.

We could have chosen a new checksum prefix entirely, invalidating all existing checksums, but felt that given the small number of repositories affected — and the provisional status of module support in 1.11 — it would be preferable to retain the existing checksums.

(Perhaps that was the wrong decision, but what's done is done.)

We do not expect any disruptions of this magnitude to occur in the future.

[VojtechVitek]

Thanks for the context, @bcmills. 👍

I understand the Go modules are still "experimental" and not stable yet. However, would you be open to adding a note to the official go 1.11 documentation, so people can find the issue & workaround easily?

[Prativa20]

go version go1.11.5 linux/amd64

go: verifying github.com/hashicorp/[email protected]: checksum mismatch
downloaded: h1:ba+UwCRuxJ7+rS+cO6JnQZUrweQjmEAkwKu9r7+HCpM=
go.sum: h1:gw2LPsqe+ri9lYBKUtpzdNgE8RzPuJ67ORTp35owxdQ=