Go: cmd/compile: large variable not initialized to 0

[zephyrtronium]

What version of Go are you using (go version)?

Initially spotted on playground (1.14.9), but also reproduces with 1.15.2 and:

$ go version
go version devel +3036b76df0 Sat Oct 10 16:06:07 2020 +0000 windows/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

PS E:\go> go env

set GO111MODULE=on

set GOARCH=amd64

set GOBIN=

set GOCACHE=C:\Users\zephyr\AppData\Local\go-build

set GOENV=C:\Users\zephyr\AppData\Roaming\go\env

set GOEXE=.exe

set GOFLAGS=

set GOHOSTARCH=amd64

set GOHOSTOS=windows

set GOINSECURE=

set GOMODCACHE=E:\go\pkg\mod

set GONOPROXY=

set GONOSUMDB=

set GOOS=windows

set GOPATH=E:\go

set GOPRIVATE=

set GOPROXY=https://proxy.golang.org,direct

set GOROOT=E:\gotip

set GOSUMDB=sum.golang.org

set GOTMPDIR=

set GOTOOLDIR=E:\gotip\pkg\tool\windows_amd64

set GCCGO=gccgo

set AR=ar

set CC=gcc

set CXX=g++

set CGO_ENABLED=1

set GOMOD=NUL

set CGO_CFLAGS=-g -O2

set CGO_CPPFLAGS=

set CGO_CXXFLAGS=-g -O2

set CGO_FFLAGS=-g -O2

set CGO_LDFLAGS=-g -O2

set PKG_CONFIG=pkg-config

set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\zephyr\AppData\Local\Temp\go-build322924098=/tmp/go-build -gno-record-gcc-switches

What did you do?

Create a large array and print the first few elements of it:

https://play.golang.org/p/66eTJdT8Ctu

What did you expect to see?

Either a slice of 0s, or a compilation error. (I was originally trying to demonstrate an error that can be reported by dowidth. An error does occur when the size of the array is greater than 2e9 but less than 1<<32, but it originates elsewhere.)

What did you see instead?

[1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 1 0 1], with other apparently random combinations of 1s and 0s for other sizes. Testing locally, I also see different combinations on the same size between my (slightly dated) copy of tip and 1.15.2.

[ianlancetaylor]

CC @randall77 @mdempsky

[mdempsky]

I think this is actually a linker issue. Looking at go tool compile -S output, it looks like we're correctly emitting code to generate a 4GiB variable. It seems like the linker is truncating the size to 32-bits somewhere. I thought it was here:

https://github.com/golang/go/blob/9499a2e10840e3878bff06e7f9a3bdaa56c5ca45/src/cmd/internal/obj/objfile.go#L334

but that code was newly merged from the dev.link branch, and the issue reproduces identically under Go 1.15 (including the compiler's -S output looking right).

/cc @cherrymui @thanm

[cherrymui]

but that code was newly merged from the dev.link branch, and the issue reproduces identically under Go 1.15 (including the compiler's -S output looking right).

Go 1.15 has the same code as well (in a different place).

Go 1.14 rejects it.

main.x: symbol too large (4294967296 bytes)
too much data in section SNOPTRBSS (over 2e+09 bytes)

We never supported symbol larger than 2GB (issue #9862), so the object file uses 32-bit for symbol size. I'll add a check before truncation.

[go101]

It looks that gc 1.15.3 rejects var x [1<<31]byte but accepts var x [1<<32]byte.

[gopherbot]

Change https://golang.org/cl/263641 mentions this issue: cmd/internal/obj: reject too large symbols

[go101]

What is the difficulty to declare very large sized arrays?
If var x = make([]byte, 1<<32) can be allocated without problem, then why to limit the length of package-level arrays?

[go101]

It looks the fix makes the old code "var x [1<<31]byte" fail to compile now. An incompatibility?

[go101]

Sorry, I forgot "var x [1<<31]byte" doesn't compile before. ;)