Go: runtime: MSpanList_Insert error using Go 1.7.1 on macOS Sierra

[sklevenz]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.7.1

What operating system and processor architecture are you using (go env)?

macOS Sierra Version 10.12

GOARCH="amd64"
GOBIN="/Users/d036242/dev/go/bin"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/d036242/dev/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.7.1/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.7.1/libexec/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/33/7ggkbf316lq01872br8pmjv8ccxxy9/T/go-build804843343=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

We are working with CloudFoundry and using Spiff and Yaml2Json programs written in Go.
https://github.com/cloudfoundry-incubator/spiff
https://github.com/bronze1man/yaml2json

What did you expect to see?

spiff file1.yml file2.yml | yaml2json

Expect to see a json document on stdout.

What did you see instead?

Since update to macOS Sierra both tools mentioned above show this MSpanList_Insert error sometimes. The problem occurs sporadically and is not reproducible. Both tools were re-compiled using Go 1.7.1.

failed MSpanList_Insert 0x464110 0x3c235d63c754c 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.MSpanList_Insert(0x405990, 0x464110)
/usr/local/go/src/runtime/mheap.c:692 +0x8f
runtime.MHeap_Alloc(0x4040c0, 0x1, 0x10000000014, 0x10409)
/usr/local/go/src/runtime/mheap.c:240 +0x66
runtime.MCentral_CacheSpan(0x40c078, 0x463fd8)
/usr/local/go/src/runtime/mcentral.c:85 +0x167
runtime.MCache_Refill(0x45f000, 0x14, 0x463fd8)
/usr/local/go/src/runtime/mcache.c:90 +0xa0

goroutine 1 [running, locked to thread]:
runtime.switchtoM()
/usr/local/go/src/runtime/asm_amd64.s:198 fp=0xc208059b38 sp=0xc208059b30
runtime.mallocgc(0x160, 0x25e740, 0x0, 0xca67)
/usr/local/go/src/runtime/malloc.go:178 +0x849 fp=0xc208059be8 sp=0xc208059b38
runtime.newarray(0x25e740, 0x1, 0x4791c79b)
/usr/local/go/src/runtime/malloc.go:365 +0xc1 fp=0xc208059c20 sp=0xc208059be8
runtime.mapassign1(0x1f4200, 0xc20803c9f0, 0xc208059d68, 0xc208059da8)
/usr/local/go/src/runtime/hashmap.go:402 +0x109 fp=0xc208059cc0 sp=0xc208059c20
text/template.addValueFuncs(0xc20803c9f0, 0xc20803c9c0)
/usr/local/go/src/text/template/funcs.go:68 +0x447 fp=0xc208059e38 sp=0xc208059cc0
text/template.createValueFuncs(0xc20803c9c0, 0xc20803c9c0)
/usr/local/go/src/text/template/funcs.go:54 +0x50 fp=0xc208059e60 sp=0xc208059e38
text/template.init()
/usr/local/go/src/text/template/funcs.go:49 +0xb91 fp=0xc208059f78 sp=0xc208059e60
github.com/codegangsta/cli.init()
/home/dev/go/spiff/src/github.com/codegangsta/cli/help.go:215 +0x59 fp=0xc208059f90 sp=0xc208059f78
main.init()
/home/dev/go/spiff/src/github.com/cloudfoundry-incubator/spiff/spiff++/spiff.go:193 +0x60 fp=0xc208059f98 sp=0xc208059f90
runtime.main()
/usr/local/go/src/runtime/proc.go:58 +0xe4 fp=0xc208059fe0 sp=0xc208059f98
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc208059fe8 sp=0xc208059fe0

[bogem]

Duplicate https://github.com/golang/go/issues/17182

[shakeel]

I see this issue was marked as a duplicate of #17182 and the problem was resolved because of GOBOOTSTRAP error while building the Go distribution. But I am seeing this panic consistently after upgrading to MacOS Sierra and trying to build fuchsia. This panic consistently occurs when the mojom tool tries to generate Go bindings. I am not building a Go distribution at all.

The mojom_tool.py is wrapper over the correct binaries, Linux or MacOS, for the mojom tool written in Go.

/usr/bin/env ../../mojo/public/tools/bindings/mojom_tool.py gen --src-root-path ../../ -I ../../ -I ../../ --output-dir /Users/shakeel/google/fuchsia/out/debug-x86-64/gen --no-gen-imports --generators legacy,c,go --gen-arg dart_ignore-package-annotations ../../mojo/public/interfaces/bindings/tests/versioning_test_client.mojom
failed MSpanList_Insert 0x4320d0 0x1d9fe141a5ec0 0x0 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.throw(0x2a1af0, 0x10)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/panic.go:530 +0x90
runtime.(_mSpanList).insert(0x3c3ec8, 0x4320d0)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/mheap.go:933 +0x293
runtime.(_mheap).freeSpanLocked(0x3c36c0, 0x4320d0, 0x100, 0x0)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/mheap.go:809 +0x4be
runtime.(_mheap).grow(0x3c36c0, 0x80, 0x0)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/mheap.go:675 +0x2a0
runtime.(_mheap).allocSpanLocked(0x3c36c0, 0x80, 0xc81ffcdc00)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/mheap.go:553 +0x4e3
runtime.(_mheap).alloc_m(0x3c36c0, 0x80, 0x100000000, 0x3c5790)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/mheap.go:437 +0x119
runtime.(_mheap).alloc.func1()
/ssd/fuchsia/buildtools/linux64/go/src/runtime/mheap.go:502 +0x41
runtime.systemstack(0xc820039ee0)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/asm_amd64.s:307 +0xab
runtime.(*mheap).alloc(0x3c36c0, 0x80, 0x10100000000, 0x1ce1c)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/mheap.go:503 +0x63
runtime.largeAlloc(0x100000, 0x1, 0x20)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/malloc.go:766 +0xb3
runtime.mallocgc.func3()
/ssd/fuchsia/buildtools/linux64/go/src/runtime/malloc.go:664 +0x33
runtime.systemstack(0xc8200016c0)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/asm_amd64.s:291 +0x79
runtime.mstart()
/ssd/fuchsia/buildtools/linux64/go/src/runtime/proc.go:1048

goroutine 1 [running]:
runtime.systemstack_switch()
/ssd/fuchsia/buildtools/linux64/go/src/runtime/asm_amd64.s:245 fp=0xc82004c968 sp=0xc82004c960
runtime.mallocgc(0x100000, 0x1c2260, 0x1, 0xc820619590)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/malloc.go:665 +0x9eb fp=0xc82004ca40 sp=0xc82004c968
runtime.newarray(0x1c2260, 0x20000, 0xc820619590)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/malloc.go:798 +0xc9 fp=0xc82004ca80 sp=0xc82004ca40
runtime.makeslice(0x1b8de0, 0x20000, 0x20000, 0x0, 0x0, 0x0)
/ssd/fuchsia/buildtools/linux64/go/src/runtime/slice.go:32 +0x165 fp=0xc82004cad0 sp=0xc82004ca80
compress/flate.(_compressor).init(0xc82061b0e0, 0x485ee8, 0xc820626230, 0x9, 0x0, 0x0)
/ssd/fuchsia/buildtools/linux64/go/src/compress/flate/deflate.go:410 +0x849 fp=0xc82004cc68 sp=0xc82004cad0
compress/flate.NewWriter(0x485ee8, 0xc820626230, 0x9, 0xa, 0x0, 0x0)
/ssd/fuchsia/buildtools/linux64/go/src/compress/flate/deflate.go:487 +0x64 fp=0xc82004cca8 sp=0xc82004cc68
compress/gzip.(_Writer).Write(0xc820644000, 0xc82063e000, 0x28d8, 0x3000, 0x0, 0x0, 0x0)
/ssd/fuchsia/buildtools/linux64/go/src/compress/gzip/gzip.go:211 +0x528 fp=0xc82004cd68 sp=0xc82004cca8
mojom/mojom_tool/serialization.translateMojomFile(0xc820018240, 0xc820612d80, 0xc820064540, 0x60, 0xc820018258, 0xc820612048, 0xc820613520, 0x0, 0xc820613560, 0xc820613580, ...)
/ssd/fuchsia/mojom/src/mojom/mojom_tool/serialization/serialization.go:243 +0x1610 fp=0xc82004d510 sp=0xc82004cd68
mojom/mojom_tool/serialization.translateDescriptor(0xc820062070, 0x1)
/ssd/fuchsia/mojom/src/mojom/mojom_tool/serialization/serialization.go:109 +0x4ca fp=0xc82004d840 sp=0xc82004d510
mojom/mojom_tool/serialization.serialize(0xc820062070, 0x1010100, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/ssd/fuchsia/mojom/src/mojom/mojom_tool/serialization/serialization.go:62 +0xc7 fp=0xc82004d8e0 sp=0xc82004d840
mojom/mojom_tool/serialization.Serialize(0xc820062070, 0xc82000a100, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/ssd/fuchsia/mojom/src/mojom/mojom_tool/serialization/serialization.go:42 +0x75 fp=0xc82004d930 sp=0xc82004d8e0
main.parse(0xc82000a1f0, 0x1, 0x1, 0xc82000e3e0, 0x2, 0x2, 0xc82004dd80, 0x0, 0x0, 0x0, ...)
/ssd/fuchsia/mojom/src/mojom/mojom_tool/parse_cmd.go:118 +0x5ba fp=0xc82004db70 sp=0xc82004d930
main.genCmd(0xc82000a100, 0x10, 0x10)
/ssd/fuchsia/mojom/src/mojom/mojom_tool/gen_cmd.go:119 +0x614 fp=0xc82004de10 sp=0xc82004db70
main.(*commandSet).RunCommand(0xc82000e140, 0xc82000a100, 0x10, 0x10)
/ssd/fuchsia/mojom/src/mojom/mojom_tool/mojom_main.go:98 +0x514 fp=0xc82004def8 sp=0xc82004de10
main.main()
/ssd/fuchsia/mojom/src/mojom/mojom_tool/mojom_main.go:36 +0x1c7 fp=0xc82004df40 sp=0xc82004def8
runtime.main()
/ssd/fuchsia/buildtools/linux64/go/src/runtime/proc.go:188 +0x2b0 fp=0xc82004df90 sp=0xc82004df40
runtime.goexit()
/ssd/fuchsia/buildtools/linux64/go/src/runtime/asm_amd64.s:1998 +0x1 fp=0xc82004df98 sp=0xc82004df90
[674/7323] ACTION //mojo/public/interfaces/binding...terfaces__generator(//build/toolchain/fuchsia:x64)
ninja: build stopped: subcommand failed.

[bogem]

@shakeel try to update or reinstall go

[sklevenz]

@shakeel Try to build with -a

go build -a ./...

After that my executable did not show this error anymore.

[shakeel]

@bogem and @sklevenz the mojom tool is precompiled by Google, not sure which version of Go. https://fuchsia.googlesource.com/mojo/public/+/master/tools/bindings/mojom_tool/bin And it looks like most of the developers writing mojo components are on Ubuntu and they are not affected by this defect, it is only on MacOS Sierra

[bogem]

@shakeel yes, this bug is only on macOS Sierra.

[madhanganesh]

I reinstalled go latest version on macOS Sierra, then this error did not appear (at least until now)

[totalys]

I got this error and I solved it removing USB hub. It does not occour if I directly plug the node in the MAC USB.

[tggagne]

I'm getting this error all the time. Less frequently using go 1.3 than 1.8.3 I upgraded to. The only "go" executable I use is the salesforce "force" cli tool.

DrozBook:fsb-maintenance tgagne$ uname -a
Darwin DrozBook.local 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64
DrozBook:fsb-maintenance tgagne$ go version
go version go1.8.3 darwin/amd64
DrozBook:fsb-maintenance tgagne$ force version
dev

[bradfitz]

@tggagne, that means the saleforce "force" tool needs to be recompiled with the fixed version of Go.

[jayunit100]

why is it stochastic ? I see it 1/3 of the time but not all the time. i think it only happens on certain retries or something.

[davecheney]

@jayunit100 this issue is closed. Please open a new issue and complete the template rather than commenting on a closed issue. Thanks.

[jwreedy]

@tggagne, I am sure you already fixed this, but for others who may run into the same issue with the force.com cli issue, as bradfitz suggested:

1. I downloaded and install "go"
2. I added "go" to my path in my .bash_profile
3. I downloaded and install Mercurial
4. from the directory where I had previously installed force, I compiled the force.com cli from the source:
   $ go get -u github.com/heroku/force