Go: cmd/go: Checksum mismatch between macOS and Windows (both on 1.11.4; first deleted go.sum)

[glyn]

Since upgrading to 1.11.4, I'm seeing module checksum mismatches between macOS and Windows (_both_ on 1.11.4).

What version of Go are you using (go version)?

On macOS:

$ go version
go version go1.11.4 darwin/amd64

On Windows:

$ go version
go version go1.11.4 windows/amd64

Does this issue reproduce with the latest release?

Yes as the issue is against 1.11.4. which is the latest release.

What operating system and processor architecture are you using (go env)?

darwin/amd64 and windows/amd64.

What did you do?

After upgrading to 1.11.4, I deleted go.sum (because of https://github.com/golang/go/issues/29278), rebuilt on macOS, and checked in the new go.sum.

Then I pulled the above change from Windows, upgraded to Go 1.11.4, and attempted a build.

What did you expect to see?

A clean build.

What did you see instead?

go: verifying github.com/knative/[email protected]: checksum mismatch
    downloaded: h1:BkBXjJb3ugETV9Jfk97Aa7aIjnhRRuI6EnfQ7du0QCU=
    go.sum:     h1:+bmR2edXNnc8l4zTZ1QEsy8R37ariWyaNGuahHSj+Tg=

Observations

Looking in the module cache on each OS, I notice that the zip files for github.com/knative/[email protected] are different sizes. Not sure whether this is expected, but if it's not expected, that could explain the checksum mismatch.

[bcmills]

Duplicate of #29278. (The symlinks are in kodata subdirectories.)

[glyn]

@bcmills Why is this a duplicate of https://github.com/golang/go/issues/29278? This issue only concerns mismatches between macOS and Windows when both are running Go 1.11.4. The other issue seems to be about differences between Go 1.11.4 and earlier versions.

[thepudds]

@bcmills is there a need to clear the module cache to get the new checksum calculation?

Quick test seems to hint that is the case. (In this example, 1.11.4 populates go.sum with _old_ checksum calculation if 1.11.1 ran first, but if you then do go clean -modcache 1.11.4 populates go.sum with _new_ checksum calculation)

# ----------------------------------------
# prep:

go get golang.org/dl/go1.11.1
go 1.11.1 download
go get golang.org/dl/go1.11.4
go 1.11.4 download
mkdir -p /tmp/scratchpad/checksum-29282
cd /tmp/scratchpad/checksum-29282
go mod init temp

# ----------------------------------------
# 1.11.1:

go clean -modcache
rm go.sum
go1.11.1 get -v github.com/knative/[email protected]
grep 'knative/build v0.2.0 h1' go.sum

# outputs:
# github.com/knative/build v0.2.0 h1:+bmR2edXNnc8l4zTZ1QEsy8R37ariWyaNGuahHSj+Tg=

# ----------------------------------------
# 1.11.4, after 1.11.1 but without cleaning module cache:

rm go.sum
go1.11.4 get -v github.com/knative/[email protected]
grep 'knative/build v0.2.0 h1' go.sum

# outputs:
# github.com/knative/build v0.2.0 h1:+bmR2edXNnc8l4zTZ1QEsy8R37ariWyaNGuahHSj+Tg=

# ----------------------------------------
# 1.11.4, but first clean module cache:

go clean -modcache
rm go.sum
go1.11.4 get -v github.com/knative/[email protected]
grep 'knative/build v0.2.0 h1' go.sum

# outputs:
# github.com/knative/build v0.2.0 h1:BkBXjJb3ugETV9Jfk97Aa7aIjnhRRuI6EnfQ7du0QCU=

[thepudds]

@glyn Can you try go clean -modcache on your Windows machine (as outlined above), and try again with 1.11.4?

[bcmills]

Yes, if you have a copy of a module that contains symlinks you'll need to run go clean -modcache to remove the incorrect version from the cache.

[glyn]

Thanks! A workaround was indeed to issue go clean -modcache on macOS and then re-create and check in a fresh go.sum file. With this file, the build on Windows succeeded. (I could alternatively have re-created go.sum on Windows, but I didn't want to do that as it's not our main development environment.)

Ideally, "modfetch" would automatically delete cache entries from old/incompatible versions of Go, but at the very least, advice should be added to the release notes for 1.11.4.