Go: SIGSEGV when calling list.List.PushBack

[zjshen14]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10.2 linux/amd64

Does this issue reproduce with the latest release?

N/A

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build689873858=/tmp/go-build -gno-record-gcc-switches"

What did you do?

We ran the executable in docker container (ubuntu) on digital ocean (centos). The code uses list.List to maintain some data.

What did you expect to see?

list.List.PushBack should work

What did you see instead?

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x68f874]

goroutine 12 [running]:
container/list.(*List).insert(...)
    /usr/local/go/src/container/list/list.go:97
container/list.(*List).insertValue(...)
    /usr/local/go/src/container/list/list.go:105
container/list.(*List).PushBack(0xc4201775f0, 0x9cce60, 0xc4201ba000, 0xc42153f9e0)
    /usr/local/go/src/container/list/list.go:140 +0x94

[AlexRouSg]

Can you provide a sample runnable program that shows this error?

[zjshen14]

@AlexRouSg this is where the panic happens: https://github.com/iotexproject/iotex-core/blob/fd5250a952d22c5c991f967f98670ef6647f62a0/trie/trie.go#L275. Hopefully this could be useful for you to diagnose.

Is it possible to be thread safe issue?

[AlexRouSg]

That is much too big to debug, if you cannot create a small program to reproduce this then chances are you're doing something wrong.

Have you tried running/building with -race enabled? https://blog.golang.org/race-detector

[zjshen14]

Yup, but unfortunately, we only encounter the issue sporadically in the distributed deployment, so that I couldn't give you a small program to reproduce it.

By -race, are you indicating it's the race condition on mutating the list?

[AlexRouSg]

The most obvious way to get your exact error message would be to call PushBack and Remove at the same time.

[AlexRouSg]

Proof of concept program below.

package main

import (
    "container/list"
    "fmt"
)

func main() {
    l := list.List{}

    ready := make(chan struct{})

    go func() {
        ready <- struct{}{}
        for i := 0; i < (1 << 20); i++ {
            l.PushBack(0)
        }
    }()

    <-ready

    for i := 0; i < (1 << 20); i++ {
        l.Remove(l.Back())
    }

    fmt.Println(l.Len())
}

[zjshen14]

Thanks for the POC @AlexRouSg