Go: cmd/compile: compiler crashes on programs with 1<<30 array key

[dotaheor]

What version of Go are you using (go version)?

go version go1.9.3 linux/amd64

Does this issue reproduce with the latest release?

yes

What did you do?

a.go:

package main

import (
    "fmt"
)

func main() {
    fmt.Println("Hello, playground")
    const k uint64 = 1 << 30
    _ = []int{k:1}
}}

go build a.go

What did you expect to see?

produce a binary in one second.

What did you see instead?

computer becomes lagging, almost hanging.
I must press CTRL-C for several times to stop the building.

[cznic]

I can reproduce the issue with go version go1.9.3 linux/amd64 when replacing the final }} with } to make the code syntactically valid.

[dotaheor]

The hang may be caused by my computer (8G memory) has not enough memory.
But I think this should only happen at run time, not compile time.

If I change the slice element type to byte, it fails to compile with the following error:

./a.go:10:15: prepwrite: bad off=1073741824 siz=1 s="".statictmp_1

[agnivade]

You are trying to create a slice of 1073741824 integers. Which would mean you are trying to get 1073741824 * 8 bytes = 8GB of memory approximately.

[dotaheor]

Yes. But I think the hang should occur at run time, instead of compile time.

[ALTree]

The second issue you reported (the fact that with byte the compiler just prints

./a.go:10:15: prepwrite: bad off=1073741824 siz=1 s="".statictmp_1

) looks more serious.

I suspect it would do the same thing with int64 on a machine with enough memory.

[ALTree]

I changed the issue's title to highlight the linker(?) crash you reported in your second post, since I believe it is more interesting that your machine getting slow on the program having the int64 array.

[agnivade]

I think it is to do with the way the slice is being initialized. []int{k:1} creates a slice of 0,0,...1. But if you change the code to make([]int, k) everything works fine. Frankly, I have never seen syntax like this - []int{k:1}.

[cznic]

FTR: foo := []T{n:1} makes foo with n+1 elements in contrast to foo := make([]t, n).

Note to the compiler guys: I guess the compiler tries to be nice wrt to initialization time and puts the images for composite literals in the text and/or data segments - triggering this issue. If that's the case I think the solution would be to detect when the composite literal is very sparse/mostly zero-values and perform instead a run time all-zero value creation and initialization of the non zero-valued items only all in code.

[odeke-em]

/cc @griesemer @mdempsky @josharian

[griesemer]

@cznic Yes, the compiler could probably be a bit smarter here - on the other hand, perhaps one shouldn't write code like this. But I agree that the compiler shouldn't just appear to be hanging.

To summarize:

1) $ go tool compile x.go
x.go:4:22: prepwrite: bad off=1073741824 siz=1 s="".statictmp_0
for x.go:

package p

func f() {
    _ = []byte{1 << 30: 0}
}

2) $ go tool compile x.go
x.go:4:21: prepwrite: bad off=8589934592 siz=8 s="".statictmp_0
for x.go (same as above, but int instead of byte):

package p

func f() {
    _ = []int{1 << 30: 0}
}

3) There's no issue with:

package p

var X = []int{1 << 30: 0}

[griesemer]

Not urgent. Moving to 1.12.

[randall77]

Related to #27447

[gopherbot]

Change https://golang.org/cl/151319 mentions this issue: cmd/compile: initialize sparse slice literals dynamically