Go-ethereum: Geth 1.8.2 in docker : fatal error: runtime: out of memory
Hi there,
I get a fatal error: runtime: out of memory after updating geth to 1.8.2 in my docker-compose file. These two screenshots are separated by 1:30 minute (attached is video)
geth-out-of-memory.zip :
After 1'30'' one can see that geth is constantly eating up memory
These are the cli args used to start geth : geth --syncmode "fast" --testnet --ws --wsaddr "0.0.0.0" --wsorigins "*" --rpcvhosts my.domain --cache 512
Here is my server RAM:
less /proc/meminfo
MemTotal: 4045092 kB
Is that related to https://github.com/ethereum/go-ethereum/issues/16244, https://github.com/ethereum/go-ethereum/issues/16243 and https://github.com/ethereum/go-ethereum/issues/16174 ?
Many thanks for your help!
System information
Geth version: 1.8.2
OS & Version: Linux Docker
Backtrace
2018-03-23T08:46:59.051466662Z INFO [03-23|08:46:58] Generating ethash verification cache epoch=101 percentage=88 elapsed=2m33.666s
2018-03-23T08:46:59.110963632Z INFO [03-23|08:46:59] Imported new state entries count=555 elapsed=48.329ms processed=3136088 pending=1406 retry=0 duplicate=0 unexpected=0
2018-03-23T08:46:59.110981772Z INFO [03-23|08:46:59] Imported new block headers count=0 elapsed=67.024ms number=2996204 hash=6010cc…4adc93 ignored=192
2018-03-23T08:46:59.114108529Z INFO [03-23|08:46:59] Imported new block receipts count=213 elapsed=96.369ms number=2994949 hash=e228fa…95c5ba size=667.78kB ignored=0
2018-03-23T08:46:59.125402457Z INFO [03-23|08:46:59] Imported new block headers count=0 elapsed=6.390ms number=2996396 hash=c260de…cbb623 ignored=192
2018-03-23T08:47:02.157967836Z INFO [03-23|08:47:01] Generating ethash verification cache epoch=101 percentage=88 elapsed=2m36.802s
2018-03-23T08:47:02.456907921Z INFO [03-23|08:47:02] Imported new block headers count=0 elapsed=91.459ms number=2996588 hash=573842…dfd9ef ignored=192
2018-03-23T08:47:02.480951206Z INFO [03-23|08:47:02] Imported new block receipts count=207 elapsed=108.148ms number=2995156 hash=3f1189…10ca8e size=831.33kB ignored=0
2018-03-23T08:47:05.249866518Z INFO [03-23|08:47:05] Generating ethash verification cache epoch=101 percentage=88 elapsed=2m39.975s
2018-03-23T08:47:05.782409059Z INFO [03-23|08:47:05] Imported new block headers count=0 elapsed=158.628ms number=2996780 hash=26e9c4…9159ad ignored=192
2018-03-23T08:47:05.849579741Z INFO [03-23|08:47:05] Imported new block headers count=0 elapsed=28.674ms number=2996972 hash=1330d6…1b885a ignored=192
2018-03-23T08:47:05.944288540Z INFO [03-23|08:47:05] Imported new block receipts count=205 elapsed=97.359ms number=2995361 hash=f9fc57…1277a2 size=984.12kB ignored=0
2018-03-23T08:47:08.583671237Z INFO [03-23|08:47:08] Generating ethash verification cache epoch=101 percentage=88 elapsed=2m43.079s
2018-03-23T08:47:10.850788138Z INFO [03-23|08:47:10] Imported new block headers count=0 elapsed=222.984ms number=2997164 hash=5ef424…021eb8 ignored=192
2018-03-23T08:47:11.534812147Z fatal error: runtime: out of memory
2018-03-23T08:47:11.598728309Z
2018-03-23T08:47:11.598741686Z runtime stack:
2018-03-23T08:47:11.598745665Z runtime.throw(0xf8b55f, 0x16)
2018-03-23T08:47:11.598749468Z /usr/local/go/src/runtime/panic.go:605 +0x95
2018-03-23T08:47:11.598753798Z runtime.sysMap(0xc47f2f0000, 0xc000000, 0xc4200ec700, 0x1bbe078)
2018-03-23T08:47:11.598757763Z /usr/local/go/src/runtime/mem_linux.go:216 +0x1d0
2018-03-23T08:47:11.598761524Z runtime.(*mheap).sysAlloc(0x1ba4700, 0xc000000, 0x7f681174f868)
2018-03-23T08:47:11.598765551Z /usr/local/go/src/runtime/malloc.go:470 +0xd7
2018-03-23T08:47:11.598769054Z runtime.(*mheap).grow(0x1ba4700, 0x6000, 0x0)
2018-03-23T08:47:11.598772527Z /usr/local/go/src/runtime/mheap.go:887 +0x60
2018-03-23T08:47:11.598776043Z runtime.(*mheap).allocSpanLocked(0x1ba4700, 0x6000, 0x1bbe088, 0xc4219a1200)
2018-03-23T08:47:11.598779731Z /usr/local/go/src/runtime/mheap.go:800 +0x334
2018-03-23T08:47:11.598783253Z runtime.(*mheap).alloc_m(0x1ba4700, 0x6000, 0x430101, 0x1b932d8)
2018-03-23T08:47:11.598787700Z /usr/local/go/src/runtime/mheap.go:666 +0x118
2018-03-23T08:47:11.598791588Z runtime.(*mheap).alloc.func1()
2018-03-23T08:47:11.598795029Z /usr/local/go/src/runtime/mheap.go:733 +0x4d
2018-03-23T08:47:11.598798424Z runtime.systemstack(0x7f681174f950)
2018-03-23T08:47:11.598801807Z /usr/local/go/src/runtime/asm_amd64.s:360 +0xab
2018-03-23T08:47:11.598805406Z runtime.(*mheap).alloc(0x1ba4700, 0x6000, 0x7f6811010101, 0x4513a5)
2018-03-23T08:47:11.598808910Z /usr/local/go/src/runtime/mheap.go:732 +0xa1
2018-03-23T08:47:11.598812325Z runtime.largeAlloc(0xc000000, 0xc433800101, 0x10000c47f1837a8)
2018-03-23T08:47:11.598815858Z /usr/local/go/src/runtime/malloc.go:827 +0x98
2018-03-23T08:47:11.598819313Z runtime.mallocgc.func1()
2018-03-23T08:47:11.598822687Z /usr/local/go/src/runtime/malloc.go:722 +0x46
2018-03-23T08:47:11.598826031Z runtime.systemstack(0xc42001f300)
2018-03-23T08:47:11.598829425Z /usr/local/go/src/runtime/asm_amd64.s:344 +0x79
2018-03-23T08:47:11.598832824Z runtime.mstart()
2018-03-23T08:47:11.598836129Z /usr/local/go/src/runtime/proc.go:1135
2018-03-23T08:47:11.598839521Z
2018-03-23T08:47:11.598842756Z goroutine 51229 [running]:
2018-03-23T08:47:11.598846156Z runtime.systemstack_switch()
2018-03-23T08:47:11.598849658Z /usr/local/go/src/runtime/asm_amd64.s:298 fp=0xc47f17eaa8 sp=0xc47f17eaa0 pc=0x46ef00
2018-03-23T08:47:11.598853232Z runtime.mallocgc(0xc000000, 0xdeec80, 0xdeadbe01, 0xc461dc8600)
2018-03-23T08:47:11.598856702Z /usr/local/go/src/runtime/malloc.go:721 +0x7ae fp=0xc47f17eb50 sp=0xc47f17eaa8 pc=0x42566e
2018-03-23T08:47:11.604469618Z runtime.makeslice(0xdeec80, 0x0, 0xc000000, 0x7f6811ace000, 0xc45930c000, 0xc456b6cd01)
2018-03-23T08:47:11.604479143Z /usr/local/go/src/runtime/slice.go:54 +0x77 fp=0xc47f17eb80 sp=0xc47f17eb50 pc=0x4586b7
2018-03-23T08:47:11.604483047Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/memdb.New(0x1834820, 0xc42049e6c0, 0xc000000, 0x0)
2018-03-23T08:47:11.630941869Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/memdb/memdb.go:470 +0xfc fp=0xc47f17ec20 sp=0xc47f17eb80 pc=0x65160c
2018-03-23T08:47:11.630972668Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mpoolGet(0xc420234680, 0x1d347, 0x0)
2018-03-23T08:47:11.630983313Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:90 +0xb5 fp=0xc47f17ec68 sp=0xc47f17ec20 pc=0x67cf45
2018-03-23T08:47:11.630992059Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).newMem(0xc420234680, 0x1d347, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.631004268Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:147 +0x23b fp=0xc47f17ed18 sp=0xc47f17ec68 pc=0x67d4cb
2018-03-23T08:47:11.631013168Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).rotateMem(0xc420234680, 0x1d347, 0x0, 0xc47f17eef0, 0xc46c9c60f0, 0x7f6810aad730)
2018-03-23T08:47:11.631021638Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:46 +0x81 fp=0xc47f17ed70 sp=0xc47f17ed18 pc=0x680c11
2018-03-23T08:47:11.631029990Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).flush.func1(0xbea54da3adac6900)
2018-03-23T08:47:11.631038726Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:102 +0x2db fp=0xc47f17ee10 sp=0xc47f17ed70 pc=0x6972fb
2018-03-23T08:47:11.631047108Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).flush(0xc420234680, 0x1d347, 0xc431089300, 0x11397, 0x0, 0x0)
2018-03-23T08:47:11.631055541Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:114 +0x171 fp=0xc47f17eee0 sp=0xc47f17ee10 pc=0x680ee1
2018-03-23T08:47:11.631063888Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).writeLocked(0xc420234680, 0xc42379da00, 0x0, 0x1, 0x0, 0x0)
2018-03-23T08:47:11.631072320Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:153 +0x6c fp=0xc47f17f088 sp=0xc47f17eee0 pc=0x6811dc
2018-03-23T08:47:11.631092190Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).Write(0xc420234680, 0xc42379da00, 0x0, 0xc4562d98a8, 0xc47f17f218)
2018-03-23T08:47:11.631096697Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:313 +0x308 fp=0xc47f17f328 sp=0xc47f17f088 pc=0x681e68
2018-03-23T08:47:11.631100461Z github.com/ethereum/go-ethereum/ethdb.(*ldbBatch).Write(0xc4603e27e0, 0x1add7, 0xc42e47ebd0)
2018-03-23T08:47:11.631111976Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/ethdb/database.go:293 +0x40 fp=0xc47f17f360 sp=0xc47f17f328 pc=0x69d960
2018-03-23T08:47:11.631118552Z github.com/ethereum/go-ethereum/core.(*BlockChain).InsertReceiptChain(0xc4205a0000, 0xc432679100, 0xd5, 0xd5, 0xc424d34000, 0xd5, 0xd5, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.631122339Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/blockchain.go:811 +0x12cc fp=0xc47f17fa60 sp=0xc47f17f360 pc=0x876fdc
2018-03-23T08:47:11.631126004Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).commitFastSyncData(0xc4200edb00, 0xc420e14800, 0xd5, 0x100, 0xc469921ab0, 0xc420e14800, 0xd5)
2018-03-23T08:47:11.631129630Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1489 +0x63d fp=0xc47f17fd70 sp=0xc47f17fa60 pc=0xa49c9d
2018-03-23T08:47:11.631133330Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFastSyncContent(0xc4200edb00, 0xc44d3d1680, 0x0, 0x0)
2018-03-23T08:47:11.631136944Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1408 +0x32e fp=0xc47f17ff60 sp=0xc47f17fd70 pc=0xa489ae
2018-03-23T08:47:11.631140569Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func7(0x8d84999300000008, 0x10086a0)
2018-03-23T08:47:11.631144150Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:469 +0x33 fp=0xc47f17ff90 sp=0xc47f17ff60 pc=0xa5ba53
2018-03-23T08:47:11.631147895Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0xc425a5ca60, 0xc44c67e600, 0xc473704680)
2018-03-23T08:47:11.631151468Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:484 +0x51 fp=0xc47f17ffc8 sp=0xc47f17ff90 pc=0xa5bad1
2018-03-23T08:47:11.631155145Z runtime.goexit()
2018-03-23T08:47:11.631158561Z /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc47f17ffd0 sp=0xc47f17ffc8 pc=0x471b31
2018-03-23T08:47:11.631162248Z created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
2018-03-23T08:47:11.631166351Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:484 +0xc7
2018-03-23T08:47:11.631170057Z
2018-03-23T08:47:11.631173347Z goroutine 1 [chan receive, 25 minutes]:
2018-03-23T08:47:11.631176907Z github.com/ethereum/go-ethereum/node.(*Node).Wait(0xc4201d4000)
2018-03-23T08:47:11.631180376Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/node/node.go:539 +0x7f
2018-03-23T08:47:11.631183936Z main.geth(0xc420216b00, 0x1006748, 0xb2d05e00)
2018-03-23T08:47:11.631187344Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/cmd/geth/main.go:220 +0x51
2018-03-23T08:47:11.631191267Z github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli%2ev1.HandleAction(0xe077e0, 0x1007c08, 0xc420216b00, 0xc420216b00, 0xc420497f40)
2018-03-23T08:47:11.631194961Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli.v1/app.go:490 +0xd2
2018-03-23T08:47:11.631201780Z github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli%2ev1.(*App).Run(0xc420234ea0, 0xc4200100a0, 0xa, 0xa, 0x0, 0x0)
2018-03-23T08:47:11.631205539Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/gopkg.in/urfave/cli.v1/app.go:264 +0x635
2018-03-23T08:47:11.631209078Z main.main()
2018-03-23T08:47:11.631212488Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/cmd/geth/main.go:208 +0x55
2018-03-23T08:47:11.631216011Z
2018-03-23T08:47:11.631219308Z goroutine 6 [chan receive, 25 minutes]:
2018-03-23T08:47:11.631222748Z github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.(*nonrecursiveTree).dispatch(0xc42005c1e0, 0xc42005c120)
2018-03-23T08:47:11.631226339Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/tree_nonrecursive.go:36 +0xc0
2018-03-23T08:47:11.631230076Z created by github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.newNonrecursiveTree
2018-03-23T08:47:11.631233610Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/tree_nonrecursive.go:29 +0x149
2018-03-23T08:47:11.631237187Z
2018-03-23T08:47:11.631240555Z goroutine 7 [chan receive, 25 minutes]:
2018-03-23T08:47:11.631244014Z github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.(*nonrecursiveTree).internal(0xc42005c1e0, 0xc42005c180)
2018-03-23T08:47:11.631247641Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/tree_nonrecursive.go:81 +0x88
2018-03-23T08:47:11.631251259Z created by github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.newNonrecursiveTree
2018-03-23T08:47:11.631254761Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/tree_nonrecursive.go:30 +0x178
2018-03-23T08:47:11.631258393Z
2018-03-23T08:47:11.631261837Z goroutine 8 [syscall, 25 minutes]:
2018-03-23T08:47:11.631265342Z os/signal.signal_recv(0x0)
2018-03-23T08:47:11.631268816Z /usr/local/go/src/runtime/sigqueue.go:131 +0xa6
2018-03-23T08:47:11.631273917Z os/signal.loop()
2018-03-23T08:47:11.631277352Z /usr/local/go/src/os/signal/signal_unix.go:22 +0x22
2018-03-23T08:47:11.631280852Z created by os/signal.init.0
2018-03-23T08:47:11.631284261Z /usr/local/go/src/os/signal/signal_unix.go:28 +0x41
2018-03-23T08:47:11.631287718Z
2018-03-23T08:47:11.631290956Z goroutine 10 [select, 25 minutes]:
2018-03-23T08:47:11.631294336Z github.com/ethereum/go-ethereum/accounts/keystore.(*watcher).loop(0xc4204445c0)
2018-03-23T08:47:11.631297833Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/accounts/keystore/watch.go:94 +0x511
2018-03-23T08:47:11.631301410Z created by github.com/ethereum/go-ethereum/accounts/keystore.(*watcher).start
2018-03-23T08:47:11.631309537Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/accounts/keystore/watch.go:52 +0x5d
2018-03-23T08:47:11.631313312Z
2018-03-23T08:47:11.631316636Z goroutine 21 [syscall, 25 minutes]:
2018-03-23T08:47:11.631320000Z syscall.Syscall6(0xe8, 0x6, 0xc42002576c, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.631335238Z /usr/local/go/src/syscall/asm_linux_amd64.s:44 +0x5
2018-03-23T08:47:11.631338557Z github.com/ethereum/go-ethereum/vendor/golang.org/x/sys/unix.EpollWait(0x6, 0xc42002576c, 0x1, 0x1, 0xffffffffffffffff, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.631342026Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/golang.org/x/sys/unix/zsyscall_linux_amd64.go:1529 +0x77
2018-03-23T08:47:11.631345512Z github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.(*inotify).loop(0xc4200a4000, 0xc4200661e0)
2018-03-23T08:47:11.631348854Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/watcher_inotify.go:194 +0x8b
2018-03-23T08:47:11.631352307Z created by github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.(*inotify).lazyinit
2018-03-23T08:47:11.631355636Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/watcher_inotify.go:134 +0x13a
2018-03-23T08:47:11.631359116Z
2018-03-23T08:47:11.631362227Z goroutine 22 [chan receive, 25 minutes]:
2018-03-23T08:47:11.631365522Z github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.(*inotify).send(0xc4200a4000, 0xc4200661e0)
2018-03-23T08:47:11.631368923Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/watcher_inotify.go:254 +0xc0
2018-03-23T08:47:11.631372398Z created by github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.(*inotify).lazyinit
2018-03-23T08:47:11.631376350Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/watcher_inotify.go:137 +0x18d
2018-03-23T08:47:11.631379882Z
2018-03-23T08:47:11.631383143Z goroutine 23 [chan receive, 25 minutes]:
2018-03-23T08:47:11.631386413Z github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.(*inotify).send(0xc4200a4000, 0xc4200661e0)
2018-03-23T08:47:11.631389793Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/watcher_inotify.go:254 +0xc0
2018-03-23T08:47:11.631393343Z created by github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify.(*inotify).lazyinit
2018-03-23T08:47:11.631396863Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/watcher_inotify.go:137 +0x18d
2018-03-23T08:47:11.631400413Z
2018-03-23T08:47:11.631403677Z goroutine 11 [select]:
2018-03-23T08:47:11.631407039Z github.com/ethereum/go-ethereum/accounts/keystore.(*KeyStore).updater(0xc42009a000)
2018-03-23T08:47:11.631410457Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/accounts/keystore/keystore.go:203 +0x104
2018-03-23T08:47:11.631417052Z created by github.com/ethereum/go-ethereum/accounts/keystore.(*KeyStore).Subscribe
2018-03-23T08:47:11.631420534Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/accounts/keystore/keystore.go:190 +0x12c
2018-03-23T08:47:11.631424266Z
2018-03-23T08:47:11.631427420Z goroutine 12 [sleep]:
2018-03-23T08:47:11.631430672Z time.Sleep(0x3b9aca00)
2018-03-23T08:47:11.631434002Z /usr/local/go/src/runtime/time.go:65 +0x130
2018-03-23T08:47:11.631437330Z github.com/ethereum/go-ethereum/accounts/usbwallet.(*Hub).updater(0xc4201d2000)
2018-03-23T08:47:11.631440780Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/accounts/usbwallet/hub.go:224 +0x3e
2018-03-23T08:47:11.631444228Z created by github.com/ethereum/go-ethereum/accounts/usbwallet.(*Hub).Subscribe
2018-03-23T08:47:11.631447557Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/accounts/usbwallet/hub.go:213 +0x12c
2018-03-23T08:47:11.631451162Z
2018-03-23T08:47:11.631454372Z goroutine 13 [sleep]:
2018-03-23T08:47:11.631457579Z time.Sleep(0x3b9aca00)
2018-03-23T08:47:11.631460824Z /usr/local/go/src/runtime/time.go:65 +0x130
2018-03-23T08:47:11.631464137Z github.com/ethereum/go-ethereum/accounts/usbwallet.(*Hub).updater(0xc4201d2140)
2018-03-23T08:47:11.631467487Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/accounts/usbwallet/hub.go:224 +0x3e
2018-03-23T08:47:11.631470863Z created by github.com/ethereum/go-ethereum/accounts/usbwallet.(*Hub).Subscribe
2018-03-23T08:47:11.631474204Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/accounts/usbwallet/hub.go:213 +0x12c
2018-03-23T08:47:11.631477662Z
2018-03-23T08:47:11.631492476Z goroutine 14 [select, 25 minutes]:
2018-03-23T08:47:11.647125205Z github.com/ethereum/go-ethereum/accounts.(*Manager).update(0xc4201a65b0)
2018-03-23T08:47:11.647139408Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/accounts/manager.go:95 +0x1d6
2018-03-23T08:47:11.647144036Z created by github.com/ethereum/go-ethereum/accounts.NewManager
2018-03-23T08:47:11.647386885Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/accounts/manager.go:68 +0x5a7
2018-03-23T08:47:11.647399812Z
2018-03-23T08:47:11.647403957Z goroutine 15 [select]:
2018-03-23T08:47:11.647407514Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc420292620)
2018-03-23T08:47:11.647411249Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:206 +0x172
2018-03-23T08:47:11.647415090Z created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
2018-03-23T08:47:11.647425802Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:237 +0x195
2018-03-23T08:47:11.647457288Z
2018-03-23T08:47:11.647462723Z goroutine 24 [select]:
2018-03-23T08:47:11.647466192Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc420234680)
2018-03-23T08:47:11.647469803Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:90 +0x119
2018-03-23T08:47:11.647473583Z created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
2018-03-23T08:47:11.647477196Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:141 +0x4bd
2018-03-23T08:47:11.647480786Z
2018-03-23T08:47:11.647484128Z goroutine 25 [select]:
2018-03-23T08:47:11.647487464Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc420234680)
2018-03-23T08:47:11.647491224Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:101 +0x13b
2018-03-23T08:47:11.647494840Z created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
2018-03-23T08:47:11.647498391Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:142 +0x4df
2018-03-23T08:47:11.647503144Z
2018-03-23T08:47:11.647506445Z goroutine 26 [select]:
2018-03-23T08:47:11.647509732Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc420234680)
2018-03-23T08:47:11.647513261Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:804 +0x2c8
2018-03-23T08:47:11.647516910Z created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
2018-03-23T08:47:11.647521542Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:148 +0x65e
2018-03-23T08:47:11.647525226Z
2018-03-23T08:47:11.647528478Z goroutine 27 [select]:
2018-03-23T08:47:11.647531848Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc420234680)
2018-03-23T08:47:11.647535409Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:751 +0x192
2018-03-23T08:47:11.647539034Z created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
2018-03-23T08:47:11.647542541Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:149 +0x680
2018-03-23T08:47:11.647546156Z
2018-03-23T08:47:11.647549441Z goroutine 28 [select, 24 minutes]:
2018-03-23T08:47:11.655849497Z github.com/ethereum/go-ethereum/core.(*ChainIndexer).updateLoop(0xc4201b23c0)
2018-03-23T08:47:11.655867808Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/chain_indexer.go:271 +0x176
2018-03-23T08:47:11.655872452Z created by github.com/ethereum/go-ethereum/core.NewChainIndexer
2018-03-23T08:47:11.655876118Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/chain_indexer.go:107 +0x25e
2018-03-23T08:47:11.655879763Z
2018-03-23T08:47:11.655883315Z goroutine 29 [select]:
2018-03-23T08:47:11.655886892Z github.com/ethereum/go-ethereum/core.(*BlockChain).update(0xc4205a0000)
2018-03-23T08:47:11.655890450Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/blockchain.go:1376 +0x141
2018-03-23T08:47:11.655894047Z created by github.com/ethereum/go-ethereum/core.NewBlockChain
2018-03-23T08:47:11.655897532Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/blockchain.go:193 +0x1162
2018-03-23T08:47:11.655901085Z
2018-03-23T08:47:11.655904458Z goroutine 30 [select, 25 minutes]:
2018-03-23T08:47:11.655907930Z github.com/ethereum/go-ethereum/core.(*ChainIndexer).eventLoop(0xc4201b23c0, 0xc4201dd8c0, 0xc4200e91a0, 0x18391a0, 0xc420244520)
2018-03-23T08:47:11.655959523Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/chain_indexer.go:189 +0x261
2018-03-23T08:47:11.655976211Z created by github.com/ethereum/go-ethereum/core.(*ChainIndexer).Start
2018-03-23T08:47:11.655980889Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/chain_indexer.go:132 +0xcf
2018-03-23T08:47:11.655984628Z
2018-03-23T08:47:11.655989315Z goroutine 31 [select]:
2018-03-23T08:47:11.657213444Z github.com/ethereum/go-ethereum/core.(*TxPool).loop(0xc4201e7dc0)
2018-03-23T08:47:11.657223079Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/tx_pool.go:283 +0x367
2018-03-23T08:47:11.657226827Z created by github.com/ethereum/go-ethereum/core.NewTxPool
2018-03-23T08:47:11.657230480Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/tx_pool.go:255 +0x5fe
2018-03-23T08:47:11.657234019Z
2018-03-23T08:47:11.657237294Z goroutine 32 [select]:
2018-03-23T08:47:11.657240640Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).qosTuner(0xc4200edb00)
2018-03-23T08:47:11.657244254Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1569 +0x352
2018-03-23T08:47:11.657247783Z created by github.com/ethereum/go-ethereum/eth/downloader.New
2018-03-23T08:47:11.657251297Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:230 +0xa78
2018-03-23T08:47:11.657254918Z
2018-03-23T08:47:11.657258148Z goroutine 33 [select]:
2018-03-23T08:47:11.657261518Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).runStateSync(0xc4200edb00, 0xc469921ab0, 0x0)
2018-03-23T08:47:11.657282554Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:125 +0x463
2018-03-23T08:47:11.657287331Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).stateFetcher(0xc4200edb00)
2018-03-23T08:47:11.657290949Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:79 +0x47
2018-03-23T08:47:11.657294477Z created by github.com/ethereum/go-ethereum/eth/downloader.New
2018-03-23T08:47:11.657297879Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:231 +0xa9d
2018-03-23T08:47:11.657301712Z
2018-03-23T08:47:11.657304985Z goroutine 34 [select, 25 minutes]:
2018-03-23T08:47:11.669823783Z github.com/ethereum/go-ethereum/miner.(*worker).update(0xc4201d2640)
2018-03-23T08:47:11.669842114Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/miner/worker.go:250 +0x309
2018-03-23T08:47:11.669847036Z created by github.com/ethereum/go-ethereum/miner.newWorker
2018-03-23T08:47:11.669850696Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/miner/worker.go:153 +0x532
2018-03-23T08:47:11.669854474Z
2018-03-23T08:47:11.669857876Z goroutine 35 [chan receive, 25 minutes]:
2018-03-23T08:47:11.669861390Z github.com/ethereum/go-ethereum/miner.(*worker).wait(0xc4201d2640)
2018-03-23T08:47:11.669865087Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/miner/worker.go:293 +0x34e
2018-03-23T08:47:11.669868751Z created by github.com/ethereum/go-ethereum/miner.newWorker
2018-03-23T08:47:11.669872192Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/miner/worker.go:155 +0x554
2018-03-23T08:47:11.669875735Z
2018-03-23T08:47:11.669879176Z goroutine 37 [select, 11 minutes]:
2018-03-23T08:47:11.669882621Z github.com/ethereum/go-ethereum/p2p/nat.Map(0x1841560, 0xc4201699c0, 0xc42056e780, 0xf78481, 0x3, 0x765f, 0x765f, 0xf86cbb, 0x12)
2018-03-23T08:47:11.669886448Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/nat/nat.go:114 +0x4cb
2018-03-23T08:47:11.669890044Z created by github.com/ethereum/go-ethereum/p2p.(*Server).Start
2018-03-23T08:47:11.669893456Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:435 +0x1007
2018-03-23T08:47:11.669896978Z
2018-03-23T08:47:11.669900375Z goroutine 66 [select]:
2018-03-23T08:47:11.669903888Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc4202929a0)
2018-03-23T08:47:11.669907464Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:206 +0x172
2018-03-23T08:47:11.669911140Z created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
2018-03-23T08:47:11.669914693Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:237 +0x195
2018-03-23T08:47:11.669938327Z
2018-03-23T08:47:11.669943072Z goroutine 69 [select]:
2018-03-23T08:47:11.669946542Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc420234820)
2018-03-23T08:47:11.669950192Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:101 +0x13b
2018-03-23T08:47:11.669953902Z created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
2018-03-23T08:47:11.669958168Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:142 +0x4df
2018-03-23T08:47:11.669961827Z
2018-03-23T08:47:11.669971288Z goroutine 68 [select, 25 minutes]:
2018-03-23T08:47:11.669975723Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc420234820)
2018-03-23T08:47:11.669979243Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:90 +0x119
2018-03-23T08:47:11.669982943Z created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
2018-03-23T08:47:11.669986452Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:141 +0x4bd
2018-03-23T08:47:11.669990044Z
2018-03-23T08:47:11.669993301Z goroutine 53295 [select]:
2018-03-23T08:47:11.669996822Z github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).VerifyHeaders.func2(0xc471e88a20, 0xc442ef4600, 0xc0, 0xc0, 0xc42f00f030, 0xc46e012060, 0xc471e81800, 0xc0, 0xc0, 0xc459694a80)
2018-03-23T08:47:11.670000585Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/consensus.go:131 +0x193
2018-03-23T08:47:11.670004163Z created by github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).VerifyHeaders
2018-03-23T08:47:11.670008340Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/consensus.go:123 +0x3b8
2018-03-23T08:47:11.670011972Z
2018-03-23T08:47:11.670015225Z goroutine 70 [select, 25 minutes]:
2018-03-23T08:47:11.670018560Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc420234820)
2018-03-23T08:47:11.670022100Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:804 +0x2c8
2018-03-23T08:47:11.670025898Z created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
2018-03-23T08:47:11.670029470Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:148 +0x65e
2018-03-23T08:47:11.670033116Z
2018-03-23T08:47:11.670036530Z goroutine 71 [select, 25 minutes]:
2018-03-23T08:47:11.670039895Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc420234820)
2018-03-23T08:47:11.670047967Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:751 +0x192
2018-03-23T08:47:11.670051763Z created by github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.openDB
2018-03-23T08:47:11.670056062Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:149 +0x680
2018-03-23T08:47:11.670059740Z
2018-03-23T08:47:11.670062965Z goroutine 72 [select, 25 minutes]:
2018-03-23T08:47:11.670066332Z github.com/ethereum/go-ethereum/p2p/discover.(*nodeDB).expirer(0xc432f78240)
2018-03-23T08:47:11.670069750Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/discover/database.go:232 +0x128
2018-03-23T08:47:11.670073323Z created by github.com/ethereum/go-ethereum/p2p/discover.(*nodeDB).ensureExpirer.func1
2018-03-23T08:47:11.670076768Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/discover/database.go:223 +0x3e
2018-03-23T08:47:11.670080259Z
2018-03-23T08:47:11.670083534Z goroutine 73 [select]:
2018-03-23T08:47:11.670086830Z github.com/ethereum/go-ethereum/p2p/discover.(*Table).loop(0xc4201d28c0)
2018-03-23T08:47:11.670090353Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:381 +0x402
2018-03-23T08:47:11.670093835Z created by github.com/ethereum/go-ethereum/p2p/discover.newTable
2018-03-23T08:47:11.670097260Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:151 +0x552
2018-03-23T08:47:11.670100830Z
2018-03-23T08:47:11.670104140Z goroutine 74 [select]:
2018-03-23T08:47:11.670107509Z github.com/ethereum/go-ethereum/p2p/discover.(*udp).loop(0xc425074ee0)
2018-03-23T08:47:11.670110967Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:386 +0x5e0
2018-03-23T08:47:11.670114486Z created by github.com/ethereum/go-ethereum/p2p/discover.newUDP
2018-03-23T08:47:11.670117969Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:263 +0x308
2018-03-23T08:47:11.670121477Z
2018-03-23T08:47:11.670124731Z goroutine 75 [runnable]:
2018-03-23T08:47:11.670128136Z internal/poll.runtime_pollWait(0x7f6811acef70, 0x72, 0x0)
2018-03-23T08:47:11.670131672Z /usr/local/go/src/runtime/netpoll.go:173 +0x57
2018-03-23T08:47:11.670135125Z internal/poll.(*pollDesc).wait(0xc4201ceb98, 0x72, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.670138670Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
2018-03-23T08:47:11.670142294Z internal/poll.(*pollDesc).waitRead(0xc4201ceb98, 0xc42defc000, 0x500, 0x500)
2018-03-23T08:47:11.670145732Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
2018-03-23T08:47:11.670152197Z internal/poll.(*FD).ReadFrom(0xc4201ceb80, 0xc42defc000, 0x500, 0x500, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.670155959Z /usr/local/go/src/internal/poll/fd_unix.go:170 +0x19a
2018-03-23T08:47:11.677785952Z net.(*netFD).readFrom(0xc4201ceb80, 0xc42defc000, 0x500, 0x500, 0x2bac2fc0e983f863, 0x83f9b1e58a5439e2, 0xdab65f96294290dd, 0x103a98ad4d0a02ff, 0x8ecdc437b1b38e57)
2018-03-23T08:47:11.677800655Z /usr/local/go/src/net/fd_unix.go:208 +0x5e
2018-03-23T08:47:11.677804958Z net.(*UDPConn).readFrom(0xc4201a9090, 0xc42defc000, 0x500, 0x500, 0xc4201b06b0, 0xc422b6fed8, 0x911fa1, 0xe39a20)
2018-03-23T08:47:11.677808928Z /usr/local/go/src/net/udpsock_posix.go:47 +0x6a
2018-03-23T08:47:11.677812593Z net.(*UDPConn).ReadFromUDP(0xc4201a9090, 0xc42defc000, 0x500, 0x500, 0x500, 0x1832d20, 0xc4201b06b0, 0x0)
2018-03-23T08:47:11.677816218Z /usr/local/go/src/net/udpsock.go:106 +0x72
2018-03-23T08:47:11.677819667Z github.com/ethereum/go-ethereum/p2p/discover.(*udp).readLoop(0xc425074ee0, 0x0)
2018-03-23T08:47:11.677823234Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:521 +0xec
2018-03-23T08:47:11.677826756Z created by github.com/ethereum/go-ethereum/p2p/discover.newUDP
2018-03-23T08:47:11.677830223Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/discover/udp.go:264 +0x33a
2018-03-23T08:47:11.677833927Z
2018-03-23T08:47:11.677840125Z goroutine 76 [IO wait, 5 minutes]:
2018-03-23T08:47:11.677843737Z internal/poll.runtime_pollWait(0x7f6811acedf0, 0x72, 0xffffffffffffffff)
2018-03-23T08:47:11.677847171Z /usr/local/go/src/runtime/netpoll.go:173 +0x57
2018-03-23T08:47:11.677850650Z internal/poll.(*pollDesc).wait(0xc425150c18, 0x72, 0xc431879d00, 0x0, 0x0)
2018-03-23T08:47:11.677854295Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
2018-03-23T08:47:11.677857748Z internal/poll.(*pollDesc).waitRead(0xc425150c18, 0xffffffffffffff00, 0x0, 0x0)
2018-03-23T08:47:11.677861208Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
2018-03-23T08:47:11.677864725Z internal/poll.(*FD).Accept(0xc425150c00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.677868201Z /usr/local/go/src/internal/poll/fd_unix.go:335 +0x1e2
2018-03-23T08:47:11.677871741Z net.(*netFD).accept(0xc425150c00, 0x145c890, 0xc44f877ea0, 0x1b91d80)
2018-03-23T08:47:11.677875212Z /usr/local/go/src/net/fd_unix.go:238 +0x42
2018-03-23T08:47:11.677878671Z net.(*TCPListener).accept(0xc4262379f8, 0xc431879f10, 0x418b3b, 0xc424f97d40)
2018-03-23T08:47:11.677882171Z /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e
2018-03-23T08:47:11.677885704Z net.(*TCPListener).Accept(0xc4262379f8, 0x0, 0xc4201d4280, 0x18454e0, 0xc465df5658)
2018-03-23T08:47:11.677889221Z /usr/local/go/src/net/tcpsock.go:247 +0x49
2018-03-23T08:47:11.677892630Z github.com/ethereum/go-ethereum/p2p.(*Server).listenLoop(0xc4201d4280)
2018-03-23T08:47:11.677903493Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:762 +0x24a
2018-03-23T08:47:11.677907443Z created by github.com/ethereum/go-ethereum/p2p.(*Server).startListening
2018-03-23T08:47:11.677911502Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:519 +0x159
2018-03-23T08:47:11.677915043Z
2018-03-23T08:47:11.677918298Z goroutine 77 [select, 11 minutes]:
2018-03-23T08:47:11.677921749Z github.com/ethereum/go-ethereum/p2p/nat.Map(0x1841560, 0xc4201699c0, 0xc42056e780, 0xf78475, 0x3, 0x765f, 0x765f, 0xf81470, 0xc)
2018-03-23T08:47:11.677925306Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/nat/nat.go:114 +0x4cb
2018-03-23T08:47:11.677928826Z github.com/ethereum/go-ethereum/p2p.(*Server).startListening.func1(0xc4201d4280, 0xc42513a540)
2018-03-23T08:47:11.677932296Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:524 +0x8b
2018-03-23T08:47:11.677935857Z created by github.com/ethereum/go-ethereum/p2p.(*Server).startListening
2018-03-23T08:47:11.677939241Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:523 +0x1fb
2018-03-23T08:47:11.677942706Z
2018-03-23T08:47:11.677946045Z goroutine 78 [select]:
2018-03-23T08:47:11.677988543Z github.com/ethereum/go-ethereum/p2p.(*Server).run(0xc4201d4280, 0x18414a0, 0xc4201d1c20)
2018-03-23T08:47:11.677995449Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:589 +0x61b
2018-03-23T08:47:11.677999190Z created by github.com/ethereum/go-ethereum/p2p.(*Server).Start
2018-03-23T08:47:11.678002754Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:504 +0x7f6
2018-03-23T08:47:11.678006282Z
2018-03-23T08:47:11.678009554Z goroutine 79 [select, 25 minutes]:
2018-03-23T08:47:11.728372458Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728386838Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728391016Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728394563Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728398103Z
2018-03-23T08:47:11.728401503Z goroutine 80 [select, 25 minutes]:
2018-03-23T08:47:11.728405025Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728408554Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728412049Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728415552Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728427646Z
2018-03-23T08:47:11.728431429Z goroutine 81 [select, 25 minutes]:
2018-03-23T08:47:11.728434931Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728438513Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728442052Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728445501Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728449021Z
2018-03-23T08:47:11.728452504Z goroutine 82 [select, 25 minutes]:
2018-03-23T08:47:11.728455956Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728459466Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728462978Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728466473Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728470010Z
2018-03-23T08:47:11.728473390Z goroutine 83 [select, 25 minutes]:
2018-03-23T08:47:11.728476754Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728480284Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728483802Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728487308Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728490821Z
2018-03-23T08:47:11.728494068Z goroutine 84 [select, 25 minutes]:
2018-03-23T08:47:11.728497385Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728501745Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728505309Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728508737Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728512265Z
2018-03-23T08:47:11.728515542Z goroutine 85 [select, 25 minutes]:
2018-03-23T08:47:11.728518909Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728522322Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728525846Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728529260Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728532763Z
2018-03-23T08:47:11.728538073Z goroutine 86 [select, 25 minutes]:
2018-03-23T08:47:11.728541511Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728544989Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728548512Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728551933Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728555416Z
2018-03-23T08:47:11.728558682Z goroutine 87 [select, 25 minutes]:
2018-03-23T08:47:11.728562012Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728565436Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728568944Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728572354Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728575863Z
2018-03-23T08:47:11.728579117Z goroutine 88 [select, 25 minutes]:
2018-03-23T08:47:11.728582497Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728586032Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728589580Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728593054Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728596583Z
2018-03-23T08:47:11.728599821Z goroutine 89 [select, 25 minutes]:
2018-03-23T08:47:11.728603196Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728606862Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728610742Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728614214Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728617756Z
2018-03-23T08:47:11.728621014Z goroutine 90 [select, 25 minutes]:
2018-03-23T08:47:11.728624344Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728627876Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728631378Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728634837Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728638410Z
2018-03-23T08:47:11.728644540Z goroutine 91 [select, 25 minutes]:
2018-03-23T08:47:11.728647987Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728651513Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728655008Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728658446Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728662024Z
2018-03-23T08:47:11.728665275Z goroutine 92 [select, 25 minutes]:
2018-03-23T08:47:11.728668670Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728672138Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728675635Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728679059Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728682565Z
2018-03-23T08:47:11.728685827Z goroutine 93 [select, 25 minutes]:
2018-03-23T08:47:11.728689147Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728692601Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728696093Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728699508Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728703020Z
2018-03-23T08:47:11.728706290Z goroutine 94 [select, 25 minutes]:
2018-03-23T08:47:11.728709621Z github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers.func1(0xc420292700)
2018-03-23T08:47:11.728713039Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:55 +0x3c2
2018-03-23T08:47:11.728716497Z created by github.com/ethereum/go-ethereum/eth.(*Ethereum).startBloomHandlers
2018-03-23T08:47:11.728720214Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/bloombits.go:53 +0x48
2018-03-23T08:47:11.728723779Z
2018-03-23T08:47:11.728727049Z goroutine 95 [select, 25 minutes]:
2018-03-23T08:47:11.728730391Z github.com/ethereum/go-ethereum/eth.(*ProtocolManager).txBroadcastLoop(0xc4201b2480)
2018-03-23T08:47:11.728733850Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/handler.go:741 +0x179
2018-03-23T08:47:11.728737343Z created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).Start
2018-03-23T08:47:11.728740774Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/handler.go:209 +0xcb
2018-03-23T08:47:11.728744223Z
2018-03-23T08:47:11.728749974Z goroutine 96 [chan receive, 25 minutes]:
2018-03-23T08:47:11.728753497Z github.com/ethereum/go-ethereum/eth.(*ProtocolManager).minedBroadcastLoop(0xc4201b2480)
2018-03-23T08:47:11.728756988Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/handler.go:730 +0x59
2018-03-23T08:47:11.728760426Z created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).Start
2018-03-23T08:47:11.728763958Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/handler.go:213 +0x15f
2018-03-23T08:47:11.728767485Z
2018-03-23T08:47:11.728770766Z goroutine 97 [select]:
2018-03-23T08:47:11.728774120Z github.com/ethereum/go-ethereum/eth.(*ProtocolManager).syncer(0xc4201b2480)
2018-03-23T08:47:11.728777579Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/sync.go:145 +0x1e8
2018-03-23T08:47:11.728781010Z created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).Start
2018-03-23T08:47:11.728784504Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/handler.go:216 +0x181
2018-03-23T08:47:11.728787984Z
2018-03-23T08:47:11.728791266Z goroutine 98 [select, 25 minutes]:
2018-03-23T08:47:11.728794652Z github.com/ethereum/go-ethereum/eth.(*ProtocolManager).txsyncLoop(0xc4201b2480)
2018-03-23T08:47:11.728798131Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/sync.go:109 +0x2bc
2018-03-23T08:47:11.728801606Z created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).Start
2018-03-23T08:47:11.728805017Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/handler.go:217 +0x1a3
2018-03-23T08:47:11.728808493Z
2018-03-23T08:47:11.728811797Z goroutine 99 [select]:
2018-03-23T08:47:11.728815134Z github.com/ethereum/go-ethereum/eth/downloader.(*PublicDownloaderAPI).eventLoop(0xc430927b80)
2018-03-23T08:47:11.728818687Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/api.go:63 +0x336
2018-03-23T08:47:11.728822211Z created by github.com/ethereum/go-ethereum/eth/downloader.NewPublicDownloaderAPI
2018-03-23T08:47:11.728825696Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/api.go:49 +0xe7
2018-03-23T08:47:11.728829240Z
2018-03-23T08:47:11.728832499Z goroutine 100 [select, 25 minutes]:
2018-03-23T08:47:11.728835938Z github.com/ethereum/go-ethereum/eth/filters.(*EventSystem).eventLoop(0xc42ea803c0)
2018-03-23T08:47:11.728839732Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/filters/filter_system.go:443 +0x846
2018-03-23T08:47:11.728843299Z created by github.com/ethereum/go-ethereum/eth/filters.NewEventSystem
2018-03-23T08:47:11.728846772Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/filters/filter_system.go:113 +0x104
2018-03-23T08:47:11.728850311Z
2018-03-23T08:47:11.728856011Z goroutine 101 [chan receive]:
2018-03-23T08:47:11.728859485Z github.com/ethereum/go-ethereum/eth/filters.(*PublicFilterAPI).timeoutLoop(0xc4203645f0)
2018-03-23T08:47:11.728862991Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/filters/api.go:83 +0x124
2018-03-23T08:47:11.728866552Z created by github.com/ethereum/go-ethereum/eth/filters.NewPublicFilterAPI
2018-03-23T08:47:11.728869991Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/filters/api.go:73 +0x1a8
2018-03-23T08:47:11.728873468Z
2018-03-23T08:47:11.728876772Z goroutine 102 [IO wait, 25 minutes]:
2018-03-23T08:47:11.728880122Z internal/poll.runtime_pollWait(0x7f6811aced30, 0x72, 0xffffffffffffffff)
2018-03-23T08:47:11.728883568Z /usr/local/go/src/runtime/netpoll.go:173 +0x57
2018-03-23T08:47:11.728887074Z internal/poll.(*pollDesc).wait(0xc432b73c98, 0x72, 0xc431c7bd00, 0x0, 0x0)
2018-03-23T08:47:11.728890523Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
2018-03-23T08:47:11.728894006Z internal/poll.(*pollDesc).waitRead(0xc432b73c98, 0xffffffffffffff00, 0x0, 0x0)
2018-03-23T08:47:11.728897494Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
2018-03-23T08:47:11.728900948Z internal/poll.(*FD).Accept(0xc432b73c80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.728904415Z /usr/local/go/src/internal/poll/fd_unix.go:335 +0x1e2
2018-03-23T08:47:11.728907829Z net.(*netFD).accept(0xc432b73c80, 0x2, 0xc424d3a000, 0x14a86e0)
2018-03-23T08:47:11.728923828Z /usr/local/go/src/net/fd_unix.go:238 +0x42
2018-03-23T08:47:11.728927330Z net.(*UnixListener).accept(0xc4250c1470, 0xf87851, 0x13, 0x3)
2018-03-23T08:47:11.728930839Z /usr/local/go/src/net/unixsock_posix.go:162 +0x32
2018-03-23T08:47:11.728934463Z net.(*UnixListener).Accept(0xc4250c1470, 0xf87851, 0x13, 0xc42fd94d40, 0x2)
2018-03-23T08:47:11.728937992Z /usr/local/go/src/net/unixsock.go:241 +0x49
2018-03-23T08:47:11.728941490Z github.com/ethereum/go-ethereum/node.(*Node).startIPC.func1(0xc4201d4000, 0x183f660, 0xc4250c1470, 0xc42def4c00)
2018-03-23T08:47:11.728945050Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/node/node.go:330 +0x107
2018-03-23T08:47:11.728948609Z created by github.com/ethereum/go-ethereum/node.(*Node).startIPC
2018-03-23T08:47:11.728952104Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/node/node.go:326 +0x392
2018-03-23T08:47:11.728955622Z
2018-03-23T08:47:11.728958871Z goroutine 103 [IO wait, 25 minutes]:
2018-03-23T08:47:11.728962346Z internal/poll.runtime_pollWait(0x7f6811acec70, 0x72, 0xffffffffffffffff)
2018-03-23T08:47:11.728966237Z /usr/local/go/src/runtime/netpoll.go:173 +0x57
2018-03-23T08:47:11.728970650Z internal/poll.(*pollDesc).wait(0xc42f9aaa98, 0x72, 0xc426295500, 0x0, 0x0)
2018-03-23T08:47:11.728974130Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
2018-03-23T08:47:11.728980845Z internal/poll.(*pollDesc).waitRead(0xc42f9aaa98, 0xffffffffffffff00, 0x0, 0x0)
2018-03-23T08:47:11.728984465Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
2018-03-23T08:47:11.728987983Z internal/poll.(*FD).Accept(0xc42f9aaa80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.728991438Z /usr/local/go/src/internal/poll/fd_unix.go:335 +0x1e2
2018-03-23T08:47:11.728994859Z net.(*netFD).accept(0xc42f9aaa80, 0x4258f8, 0x30, 0xee68c0)
2018-03-23T08:47:11.728998353Z /usr/local/go/src/net/fd_unix.go:238 +0x42
2018-03-23T08:47:11.729001794Z net.(*TCPListener).accept(0xc4250d8fa0, 0x525ada, 0xee68c0, 0xc4329715f0)
2018-03-23T08:47:11.729005229Z /usr/local/go/src/net/tcpsock_posix.go:136 +0x2e
2018-03-23T08:47:11.729008670Z net.(*TCPListener).Accept(0xc4250d8fa0, 0xc42001c018, 0xe30700, 0x1802b20, 0xf5a0e0)
2018-03-23T08:47:11.729012232Z /usr/local/go/src/net/tcpsock.go:247 +0x49
2018-03-23T08:47:11.741833838Z net/http.(*Server).Serve(0xc4250a7ee0, 0x183f620, 0xc4250d8fa0, 0x0, 0x0)
2018-03-23T08:47:11.741845036Z /usr/local/go/src/net/http/server.go:2695 +0x1b2
2018-03-23T08:47:11.741849281Z created by github.com/ethereum/go-ethereum/node.(*Node).startWS
2018-03-23T08:47:11.741852810Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/node/node.go:449 +0x570
2018-03-23T08:47:11.741856534Z
2018-03-23T08:47:11.741859915Z goroutine 104 [chan receive, 25 minutes]:
2018-03-23T08:47:11.754164915Z github.com/ethereum/go-ethereum/cmd/utils.StartNode.func1(0xc4201d4000)
2018-03-23T08:47:11.754177844Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/cmd/utils/cmd.go:70 +0x105
2018-03-23T08:47:11.754182456Z created by github.com/ethereum/go-ethereum/cmd/utils.StartNode
2018-03-23T08:47:11.754186266Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/cmd/utils/cmd.go:66 +0xb7
2018-03-23T08:47:11.754190265Z
2018-03-23T08:47:11.754193715Z goroutine 105 [chan receive, 25 minutes]:
2018-03-23T08:47:11.754197281Z main.startNode.func1(0xc4201d4000, 0xc4250a2f60)
2018-03-23T08:47:11.754200777Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/cmd/geth/main.go:260 +0x32d
2018-03-23T08:47:11.754204416Z created by main.startNode
2018-03-23T08:47:11.754207862Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/cmd/geth/main.go:245 +0x28b
2018-03-23T08:47:11.754211416Z
2018-03-23T08:47:11.754214783Z goroutine 106 [semacquire, 25 minutes]:
2018-03-23T08:47:11.768352719Z sync.runtime_notifyListWait(0xc42fe00340, 0xc400000000)
2018-03-23T08:47:11.768364831Z /usr/local/go/src/runtime/sema.go:507 +0x110
2018-03-23T08:47:11.768368954Z sync.(*Cond).Wait(0xc42fe00330)
2018-03-23T08:47:11.768372483Z /usr/local/go/src/sync/cond.go:56 +0x80
2018-03-23T08:47:11.768382743Z io.(*pipe).read(0xc42fe00300, 0xc4260bbe00, 0x200, 0x200, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.768386753Z /usr/local/go/src/io/pipe.go:47 +0xc6
2018-03-23T08:47:11.780343010Z io.(*PipeReader).Read(0xc4250d8fa8, 0xc4260bbe00, 0x200, 0x200, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.780372685Z /usr/local/go/src/io/pipe.go:130 +0x4c
2018-03-23T08:47:11.791266020Z encoding/json.(*Decoder).refill(0xc4304e7180, 0xe335a0, 0x6d6fd7c8bd901d9d)
2018-03-23T08:47:11.791320447Z /usr/local/go/src/encoding/json/stream.go:152 +0x114
2018-03-23T08:47:11.791328107Z encoding/json.(*Decoder).readValue(0xc4304e7180, 0x0, 0x0, 0x4258f8)
2018-03-23T08:47:11.791332010Z /usr/local/go/src/encoding/json/stream.go:128 +0x275
2018-03-23T08:47:11.791335628Z encoding/json.(*Decoder).Decode(0xc4304e7180, 0xe49140, 0xc42fd94d80, 0x0, 0x0)
2018-03-23T08:47:11.791339312Z /usr/local/go/src/encoding/json/stream.go:57 +0x78
2018-03-23T08:47:11.806709674Z github.com/ethereum/go-ethereum/rpc.(*jsonCodec).ReadRequestHeaders(0xc420364c80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.806800614Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/json.go:126 +0xeb
2018-03-23T08:47:11.806807213Z github.com/ethereum/go-ethereum/rpc.(*Server).readRequest(0xc430927bc0, 0x1845980, 0xc420364c80, 0x1, 0xe98a80, 0xc4212a0980, 0x1840d20, 0xc432971740, 0x11)
2018-03-23T08:47:11.806811210Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/server.go:378 +0x5f
2018-03-23T08:47:11.806815133Z github.com/ethereum/go-ethereum/rpc.(*Server).serveRequest(0xc430927bc0, 0x1845980, 0xc420364c80, 0x4000000000000, 0x3, 0x0, 0x0)
2018-03-23T08:47:11.806819014Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/server.go:162 +0x2e2
2018-03-23T08:47:11.806822846Z github.com/ethereum/go-ethereum/rpc.(*Server).ServeCodec(0xc430927bc0, 0x1845980, 0xc420364c80, 0x3)
2018-03-23T08:47:11.806826530Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/server.go:218 +0x7c
2018-03-23T08:47:11.806830094Z created by github.com/ethereum/go-ethereum/rpc.DialInProc.func1
2018-03-23T08:47:11.806833677Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/inproc.go:29 +0x2c1
2018-03-23T08:47:11.806837370Z
2018-03-23T08:47:11.806863942Z goroutine 107 [select, 25 minutes]:
2018-03-23T08:47:11.806869466Z github.com/ethereum/go-ethereum/rpc.(*Client).dispatch(0xc425153d50, 0x1845600, 0xc42511b720)
2018-03-23T08:47:11.806873099Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/client.go:500 +0x440
2018-03-23T08:47:11.806876743Z created by github.com/ethereum/go-ethereum/rpc.newClient
2018-03-23T08:47:11.806880209Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/client.go:203 +0x358
2018-03-23T08:47:11.806884719Z
2018-03-23T08:47:11.806888287Z goroutine 108 [semacquire, 25 minutes]:
2018-03-23T08:47:11.807019890Z sync.runtime_notifyListWait(0xc42fe00400, 0xc400000000)
2018-03-23T08:47:11.807026178Z /usr/local/go/src/runtime/sema.go:507 +0x110
2018-03-23T08:47:11.807029794Z sync.(*Cond).Wait(0xc42fe003f0)
2018-03-23T08:47:11.807033354Z /usr/local/go/src/sync/cond.go:56 +0x80
2018-03-23T08:47:11.807036786Z io.(*pipe).read(0xc42fe003c0, 0xc4260bbc00, 0x200, 0x200, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.807040268Z /usr/local/go/src/io/pipe.go:47 +0xc6
2018-03-23T08:47:11.807044095Z io.(*PipeReader).Read(0xc4250d8fb8, 0xc4260bbc00, 0x200, 0x200, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.807047730Z /usr/local/go/src/io/pipe.go:130 +0x4c
2018-03-23T08:47:11.807051112Z encoding/json.(*Decoder).refill(0xc4304e7340, 0xdd3d80, 0x40000040)
2018-03-23T08:47:11.807054727Z /usr/local/go/src/encoding/json/stream.go:152 +0x114
2018-03-23T08:47:11.807058152Z encoding/json.(*Decoder).readValue(0xc4304e7340, 0x0, 0x0, 0x4258f8)
2018-03-23T08:47:11.807064085Z /usr/local/go/src/encoding/json/stream.go:128 +0x275
2018-03-23T08:47:11.807067768Z encoding/json.(*Decoder).Decode(0xc4304e7340, 0xe49140, 0xc42e4bc520, 0x4793d440fd42a1f0, 0x32c53a2ad808c861)
2018-03-23T08:47:11.807071425Z /usr/local/go/src/encoding/json/stream.go:57 +0x78
2018-03-23T08:47:11.807074925Z github.com/ethereum/go-ethereum/rpc.(*Client).read.func1(0xea0000, 0xc42e4bc540, 0xc42511b720, 0x7f68112eba60, 0xc42511b720)
2018-03-23T08:47:11.807078540Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/client.go:641 +0x79
2018-03-23T08:47:11.807112683Z github.com/ethereum/go-ethereum/rpc.(*Client).read(0xc425153d50, 0x1845600, 0xc42511b720, 0x0, 0x0)
2018-03-23T08:47:11.807119029Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/client.go:654 +0x152
2018-03-23T08:47:11.807122750Z created by github.com/ethereum/go-ethereum/rpc.(*Client).dispatch
2018-03-23T08:47:11.807126215Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/client.go:476 +0x71
2018-03-23T08:47:11.807129850Z
2018-03-23T08:47:11.807133143Z goroutine 53296 [copystack]:
2018-03-23T08:47:11.818714191Z github.com/ethereum/go-ethereum/vendor/github.com/golang/snappy.encodeBlock(0xc442c52c02, 0x12c7, 0x12c7, 0xc459378000, 0xfff, 0x1e00, 0x0)
2018-03-23T08:47:11.818727948Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/golang/snappy/encode_amd64.s:265 +0x38a
2018-03-23T08:47:11.818732490Z github.com/ethereum/go-ethereum/vendor/github.com/golang/snappy.Encode(0xc442c52c00, 0x12c9, 0x12c9, 0x0, 0x0, 0x0, 0x0, 0x7f6811ace7f8, 0xc43068bc00)
2018-03-23T08:47:11.818736372Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/golang/snappy/encode.go:37 +0x1b2
2018-03-23T08:47:11.818740187Z github.com/ethereum/go-ethereum/p2p.(*rlpxFrameRW).WriteMsg(0xc420012540, 0x1f, 0xfff, 0x1833820, 0xc461dc82d0, 0x0, 0x0, 0x0, 0xffc, 0xc42001e070)
2018-03-23T08:47:11.818743904Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:609 +0x10b
2018-03-23T08:47:11.818768866Z github.com/ethereum/go-ethereum/p2p.(*rlpx).WriteMsg(0xc46407aea0, 0x1f, 0xfff, 0x1833820, 0xc461dc82d0, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.818773867Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:102 +0x102
2018-03-23T08:47:11.830225080Z github.com/ethereum/go-ethereum/p2p.(*conn).WriteMsg(0xc42dfd5400, 0x1f, 0xfff, 0x1833820, 0xc461dc82d0, 0x0, 0x0, 0x0, 0x7c, 0x1b931a0)
2018-03-23T08:47:11.830238502Z <autogenerated>:1 +0x64
2018-03-23T08:47:11.830326513Z github.com/ethereum/go-ethereum/p2p.(*protoRW).WriteMsg(0xc4200125b0, 0x1f, 0xfff, 0x1833820, 0xc461dc82d0, 0x0, 0x0, 0x0, 0xc427bb7710, 0x60f176)
2018-03-23T08:47:11.830333539Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/peer.go:390 +0x1d4
2018-03-23T08:47:11.830461992Z github.com/ethereum/go-ethereum/p2p.Send(0x7f681115f490, 0xc4200125b0, 0xf, 0xdd7fc0, 0xc45930a000, 0x2, 0x0)
2018-03-23T08:47:11.830470823Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/message.go:97 +0x10f
2018-03-23T08:47:11.830545831Z github.com/ethereum/go-ethereum/eth.(*peer).RequestReceipts(0xc45d2af320, 0xc4668fd000, 0x7c, 0x7c, 0x6, 0x6)
2018-03-23T08:47:11.830553919Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/peer.go:228 +0x1d2
2018-03-23T08:47:11.841833005Z created by github.com/ethereum/go-ethereum/eth/downloader.(*peerConnection).FetchReceipts
2018-03-23T08:47:11.841846054Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/peer.go:203 +0x24b
2018-03-23T08:47:11.841850543Z
2018-03-23T08:47:11.841854113Z goroutine 51228 [chan receive]:
2018-03-23T08:47:11.841966765Z github.com/ethereum/go-ethereum/core.(*HeaderChain).ValidateHeaderChain(0xc4203103f0, 0xc442ef4600, 0xc0, 0xc0, 0x64, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.841973806Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/headerchain.go:245 +0x9db
2018-03-23T08:47:11.841977686Z github.com/ethereum/go-ethereum/core.(*BlockChain).InsertHeaderChain(0xc4205a0000, 0xc442ef4600, 0xc0, 0xc0, 0x64, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.842019166Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/blockchain.go:1439 +0xbe
2018-03-23T08:47:11.842061823Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processHeaders(0xc4200edb00, 0x2dbbad, 0x5053f1, 0xc425540580, 0x0, 0x0)
2018-03-23T08:47:11.842100404Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1265 +0x6c9
2018-03-23T08:47:11.842144319Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func6(0x8a35cb0d00000008, 0x10086a0)
2018-03-23T08:47:11.842150149Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:466 +0x49
2018-03-23T08:47:11.842193338Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0xc425a5ca60, 0xc44c67e600, 0xc45b7278f0)
2018-03-23T08:47:11.842267413Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:484 +0x51
2018-03-23T08:47:11.842275929Z created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
2018-03-23T08:47:11.842335373Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:484 +0xc7
2018-03-23T08:47:11.842341600Z
2018-03-23T08:47:11.842345130Z goroutine 50636 [IO wait]:
2018-03-23T08:47:11.842348665Z internal/poll.runtime_pollWait(0x7f6811ace7f0, 0x72, 0x0)
2018-03-23T08:47:11.842389449Z /usr/local/go/src/runtime/netpoll.go:173 +0x57
2018-03-23T08:47:11.842431649Z internal/poll.(*pollDesc).wait(0xc45c5a3a98, 0x72, 0xffffffffffffff00, 0x1837d60, 0x1830b10)
2018-03-23T08:47:11.842469207Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
2018-03-23T08:47:11.842519905Z internal/poll.(*pollDesc).waitRead(0xc45c5a3a98, 0xc457151000, 0x20, 0x20)
2018-03-23T08:47:11.842561288Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
2018-03-23T08:47:11.842602349Z internal/poll.(*FD).Read(0xc45c5a3a80, 0xc457151020, 0x20, 0x20, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.842660309Z /usr/local/go/src/internal/poll/fd_unix.go:126 +0x18a
2018-03-23T08:47:11.842666121Z net.(*netFD).Read(0xc45c5a3a80, 0xc457151020, 0x20, 0x20, 0x100190100507633, 0x0, 0x37c44ac0f11)
2018-03-23T08:47:11.842705878Z /usr/local/go/src/net/fd_unix.go:202 +0x52
2018-03-23T08:47:11.842781504Z net.(*conn).Read(0xc4615f7198, 0xc457151020, 0x20, 0x20, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.842787301Z /usr/local/go/src/net/net.go:176 +0x6d
2018-03-23T08:47:11.842917187Z io.ReadAtLeast(0x7f681115f418, 0xc4615f7198, 0xc457151020, 0x20, 0x20, 0x20, 0xf50b20, 0x168be3cb400, 0x7f681115f418)
2018-03-23T08:47:11.842959666Z /usr/local/go/src/io/io.go:309 +0x86
2018-03-23T08:47:11.843021872Z io.ReadFull(0x7f681115f418, 0xc4615f7198, 0xc457151020, 0x20, 0x20, 0x20, 0x0, 0x6fc2392f0)
2018-03-23T08:47:11.843027515Z /usr/local/go/src/io/io.go:327 +0x58
2018-03-23T08:47:11.843069613Z github.com/ethereum/go-ethereum/p2p.(*rlpxFrameRW).ReadMsg(0xc420012540, 0xbea54dab29d38da1, 0x168be3cb46a, 0x1b91d80, 0x0, 0x0, 0x1b91d80, 0x161c218e936, 0x1b91d80, 0x20002)
2018-03-23T08:47:11.843111377Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:656 +0x100
2018-03-23T08:47:11.843204185Z github.com/ethereum/go-ethereum/p2p.(*rlpx).ReadMsg(0xc46407aea0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.843210441Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/rlpx.go:95 +0x148
2018-03-23T08:47:11.843214072Z github.com/ethereum/go-ethereum/p2p.(*Peer).readLoop(0xc4207f1320, 0xc4207f1500)
2018-03-23T08:47:11.843217409Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/peer.go:256 +0xad
2018-03-23T08:47:11.843265046Z created by github.com/ethereum/go-ethereum/p2p.(*Peer).run
2018-03-23T08:47:11.843276648Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/peer.go:194 +0xf2
2018-03-23T08:47:11.843319447Z
2018-03-23T08:47:11.843325154Z goroutine 52524 [select]:
2018-03-23T08:47:11.843391457Z net.(*netFD).connect.func2(0x1840ce0, 0xc4299c5020, 0xc460bfd880, 0xc4415057a0, 0xc441505740)
2018-03-23T08:47:11.843467002Z /usr/local/go/src/net/fd_unix.go:129 +0xf2
2018-03-23T08:47:11.843473633Z created by net.(*netFD).connect
2018-03-23T08:47:11.843556807Z /usr/local/go/src/net/fd_unix.go:128 +0x2a3
2018-03-23T08:47:11.843564259Z
2018-03-23T08:47:11.843569481Z goroutine 51327 [chan receive]:
2018-03-23T08:47:11.857548779Z github.com/ethereum/go-ethereum/eth/downloader.(*stateSync).Wait(...)
2018-03-23T08:47:11.857561577Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:261
2018-03-23T08:47:11.857566032Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFastSyncContent.func1(0xc47058b8f0, 0xc4200edb00)
2018-03-23T08:47:11.857569760Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1361 +0x44
2018-03-23T08:47:11.857633259Z created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).processFastSyncContent
2018-03-23T08:47:11.857640086Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1360 +0x11b
2018-03-23T08:47:11.857643804Z
2018-03-23T08:47:11.857647101Z goroutine 51220 [chan receive]:
2018-03-23T08:47:11.857706805Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync(0xc4200edb00, 0xc451b11a40, 0x5, 0x8, 0x5, 0xc451b11a40)
2018-03-23T08:47:11.857714938Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:495 +0x14c
2018-03-23T08:47:11.857757122Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer(0xc4200edb00, 0xc46c4de870, 0x3db76b1cd021dffd, 0x509272f376475aa5, 0x3e97b2fe6214b12, 0x2209be1bde1c5ab1, 0xc425540580, 0x0, 0x0)
2018-03-23T08:47:11.857800634Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:473 +0x6a3
2018-03-23T08:47:11.857911767Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).synchronise(0xc4200edb00, 0xc45fb89890, 0x10, 0x3db76b1cd021dffd, 0x509272f376475aa5, 0x3e97b2fe6214b12, 0x2209be1bde1c5ab1, 0xc425540580, 0x1, 0x0, ...)
2018-03-23T08:47:11.858132402Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:398 +0x3e2
2018-03-23T08:47:11.858141194Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).Synchronise(0xc4200edb00, 0xc45fb89890, 0x10, 0x3db76b1cd021dffd, 0x509272f376475aa5, 0x3e97b2fe6214b12, 0x2209be1bde1c5ab1, 0xc425540580, 0x1, 0x4cd806f3bd76ea57, ...)
2018-03-23T08:47:11.858145471Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:316 +0x97
2018-03-23T08:47:11.858149332Z github.com/ethereum/go-ethereum/eth.(*ProtocolManager).synchronise(0xc4201b2480, 0xc45d2af320)
2018-03-23T08:47:11.858184686Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/sync.go:192 +0x263
2018-03-23T08:47:11.858190805Z created by github.com/ethereum/go-ethereum/eth.(*ProtocolManager).syncer
2018-03-23T08:47:11.858194315Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/sync.go:155 +0x23c
2018-03-23T08:47:11.858198010Z
2018-03-23T08:47:11.858201544Z goroutine 50637 [select]:
2018-03-23T08:47:11.858205086Z github.com/ethereum/go-ethereum/p2p.(*Peer).pingLoop(0xc4207f1320)
2018-03-23T08:47:11.858208752Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/peer.go:240 +0x182
2018-03-23T08:47:11.858212322Z created by github.com/ethereum/go-ethereum/p2p.(*Peer).run
2018-03-23T08:47:11.858215903Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/peer.go:195 +0x117
2018-03-23T08:47:11.858219430Z
2018-03-23T08:47:11.858222921Z goroutine 50638 [select]:
2018-03-23T08:47:11.858226353Z github.com/ethereum/go-ethereum/p2p.(*protoRW).ReadMsg(0xc4200125b0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.858229882Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/peer.go:403 +0x148
2018-03-23T08:47:11.858233546Z github.com/ethereum/go-ethereum/eth.(*ProtocolManager).handleMsg(0xc4201b2480, 0xc45d2af320, 0x0, 0x0)
2018-03-23T08:47:11.858237109Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/handler.go:320 +0xbd
2018-03-23T08:47:11.858247837Z github.com/ethereum/go-ethereum/eth.(*ProtocolManager).handle(0xc4201b2480, 0xc45d2af320, 0x0, 0x0)
2018-03-23T08:47:11.858372034Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/handler.go:309 +0x8ae
2018-03-23T08:47:11.858379379Z github.com/ethereum/go-ethereum/eth.NewProtocolManager.func1(0xc4207f1320, 0x18394a0, 0xc4200125b0, 0x0, 0x0)
2018-03-23T08:47:11.858383070Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/handler.go:142 +0x1f2
2018-03-23T08:47:11.858386705Z github.com/ethereum/go-ethereum/p2p.(*Peer).startProtocols.func1(0xc4200125b0, 0xc4207f1320, 0x18394a0, 0xc4200125b0)
2018-03-23T08:47:11.858390300Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/peer.go:349 +0x66
2018-03-23T08:47:11.858394005Z created by github.com/ethereum/go-ethereum/p2p.(*Peer).startProtocols
2018-03-23T08:47:11.858397475Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/peer.go:348 +0x228
2018-03-23T08:47:11.858401065Z
2018-03-23T08:47:11.858404431Z goroutine 150 [select]:
2018-03-23T08:47:11.858418833Z github.com/ethereum/go-ethereum/eth/fetcher.(*Fetcher).loop(0xc42045f6c0)
2018-03-23T08:47:11.858564537Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/fetcher/fetcher.go:316 +0x804
2018-03-23T08:47:11.858571726Z created by github.com/ethereum/go-ethereum/eth/fetcher.(*Fetcher).Start
2018-03-23T08:47:11.858580834Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/fetcher/fetcher.go:178 +0x3f
2018-03-23T08:47:11.858584661Z
2018-03-23T08:47:11.871049349Z goroutine 151 [select, 25 minutes, locked to thread]:
2018-03-23T08:47:11.873029167Z runtime.gopark(0x1008558, 0x0, 0xf7b290, 0x6, 0x18, 0x1)
2018-03-23T08:47:11.873038316Z /usr/local/go/src/runtime/proc.go:287 +0x12c
2018-03-23T08:47:11.873042118Z runtime.selectgo(0xc421d58f50, 0xc424d3c120)
2018-03-23T08:47:11.873045582Z /usr/local/go/src/runtime/select.go:395 +0x1149
2018-03-23T08:47:11.873049197Z runtime.ensureSigM.func1()
2018-03-23T08:47:11.873052672Z /usr/local/go/src/runtime/signal_unix.go:511 +0x220
2018-03-23T08:47:11.873056210Z runtime.goexit()
2018-03-23T08:47:11.873059734Z /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1
2018-03-23T08:47:11.873063193Z
2018-03-23T08:47:11.873066709Z goroutine 52758 [IO wait]:
2018-03-23T08:47:11.873070148Z internal/poll.runtime_pollWait(0x7f6811acea30, 0x77, 0xc42001e000)
2018-03-23T08:47:11.873073756Z /usr/local/go/src/runtime/netpoll.go:173 +0x57
2018-03-23T08:47:11.873077351Z internal/poll.(*pollDesc).wait(0xc460bfd898, 0x77, 0xc460375600, 0x4497fb, 0xc4603756a0)
2018-03-23T08:47:11.873080858Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
2018-03-23T08:47:11.873084324Z internal/poll.(*pollDesc).waitWrite(0xc460bfd898, 0xc460375600, 0xc4603756d8, 0x5d0d33)
2018-03-23T08:47:11.873087914Z /usr/local/go/src/internal/poll/fd_poll_runtime.go:94 +0x3d
2018-03-23T08:47:11.873091632Z internal/poll.(*FD).WaitWrite(0xc460bfd880, 0x1007d20, 0x1840ce0)
2018-03-23T08:47:11.874877244Z /usr/local/go/src/internal/poll/fd_unix.go:403 +0x37
2018-03-23T08:47:11.874882178Z net.(*netFD).connect(0xc460bfd880, 0x1840ce0, 0xc4299c5020, 0x0, 0x0, 0x1835c20, 0xc4466d57a0, 0x0, 0x0, 0x0, ...)
2018-03-23T08:47:11.874885801Z /usr/local/go/src/net/fd_unix.go:152 +0x2b6
2018-03-23T08:47:11.874889280Z net.(*netFD).dial(0xc460bfd880, 0x1840ce0, 0xc4299c5020, 0x1844400, 0x0, 0x1844400, 0xc44595bdd0, 0xc4603758f8, 0x5ee9be)
2018-03-23T08:47:11.874892976Z /usr/local/go/src/net/sock_posix.go:142 +0xe9
2018-03-23T08:47:11.874896466Z net.socket(0x1840ce0, 0xc4299c5020, 0xf78475, 0x3, 0x2, 0x1, 0x0, 0x0, 0x1844400, 0x0, ...)
2018-03-23T08:47:11.874899949Z /usr/local/go/src/net/sock_posix.go:93 +0x1a5
2018-03-23T08:47:11.874903346Z net.internetSocket(0x1840ce0, 0xc4299c5020, 0xf78475, 0x3, 0x1844400, 0x0, 0x1844400, 0xc44595bdd0, 0x1, 0x0, ...)
2018-03-23T08:47:11.874906916Z /usr/local/go/src/net/ipsock_posix.go:141 +0x129
2018-03-23T08:47:11.874910351Z net.doDialTCP(0x1840ce0, 0xc4299c5020, 0xf78475, 0x3, 0x0, 0xc44595bdd0, 0x1bbc348, 0x0, 0x0)
2018-03-23T08:47:11.874913838Z /usr/local/go/src/net/tcpsock_posix.go:62 +0xb9
2018-03-23T08:47:11.874923497Z net.dialTCP(0x1840ce0, 0xc4299c5020, 0xf78475, 0x3, 0x0, 0xc44595bdd0, 0xbea54da533bf0936, 0x16331a54035, 0x1b91d80)
2018-03-23T08:47:11.874927429Z /usr/local/go/src/net/tcpsock_posix.go:58 +0xe4
2018-03-23T08:47:11.874930899Z net.dialSingle(0x1840ce0, 0xc4299c5020, 0xc460bfd800, 0x183a320, 0xc44595bdd0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.874934509Z /usr/local/go/src/net/dial.go:547 +0x3e2
2018-03-23T08:47:11.874939408Z net.dialSerial(0x1840ce0, 0xc4299c5020, 0xc460bfd800, 0xc4262d1d80, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.874943058Z /usr/local/go/src/net/dial.go:515 +0x247
2018-03-23T08:47:11.874946556Z net.(*Dialer).DialContext(0xc4200e9aa0, 0x1840ca0, 0xc42001c018, 0xf78475, 0x3, 0xc46098e040, 0x14, 0x0, 0x0, 0x0, ...)
2018-03-23T08:47:11.874950638Z /usr/local/go/src/net/dial.go:397 +0x6ee
2018-03-23T08:47:11.874954055Z net.(*Dialer).Dial(0xc4200e9aa0, 0xf78475, 0x3, 0xc46098e040, 0x14, 0x30, 0xc427bbb468, 0x7f6810f09018, 0x0)
2018-03-23T08:47:11.874957734Z /usr/local/go/src/net/dial.go:320 +0x75
2018-03-23T08:47:11.874961111Z github.com/ethereum/go-ethereum/p2p.TCPDialer.Dial(0xc4200e9aa0, 0xc45f8a0d20, 0x30, 0xeb4dc0, 0x92bd5695270e9801, 0xc444fbc4b0)
2018-03-23T08:47:11.874964797Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/dial.go:65 +0xb6
2018-03-23T08:47:11.874968351Z github.com/ethereum/go-ethereum/p2p.(*dialTask).dial(0xc444fbd290, 0xc4201d4280, 0xc45f8a0d20, 0x6c238827c1821aea, 0xe8b547d32612822c)
2018-03-23T08:47:11.874972019Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/dial.go:349 +0x48
2018-03-23T08:47:11.874975675Z github.com/ethereum/go-ethereum/p2p.(*dialTask).Do(0xc444fbd290, 0xc4201d4280)
2018-03-23T08:47:11.874979237Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/dial.go:297 +0x58
2018-03-23T08:47:11.874982727Z github.com/ethereum/go-ethereum/p2p.(*Server).run.func2.1(0x1833620, 0xc444fbd290, 0xc4201d4280, 0xc425002ae0)
2018-03-23T08:47:11.874986230Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:570 +0x3b
2018-03-23T08:47:11.874989705Z created by github.com/ethereum/go-ethereum/p2p.(*Server).run.func2
2018-03-23T08:47:11.874993145Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:570 +0x1b3
2018-03-23T08:47:11.874996651Z
2018-03-23T08:47:11.874999888Z goroutine 51226 [select]:
2018-03-23T08:47:11.875003301Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchParts(0xc4200edb00, 0x1832d20, 0xc4201b1460, 0xc4200e95c0, 0xc4583c1360, 0xc420236070, 0xc47f183eb0, 0xc47f183f00, 0xc47f183ef0, 0xc47f183ee0, ...)
2018-03-23T08:47:11.875007129Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1004 +0x23f
2018-03-23T08:47:11.875010683Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchBodies(0xc4200edb00, 0x2dacad, 0xc427bb9f98, 0xa283d2101012a6f1)
2018-03-23T08:47:11.875014303Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:933 +0x33e
2018-03-23T08:47:11.875020887Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func4(0x9b15080b00000008, 0x10086a0)
2018-03-23T08:47:11.875024895Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:464 +0x37
2018-03-23T08:47:11.875028550Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0xc425a5ca60, 0xc44c67e600, 0xc473704640)
2018-03-23T08:47:11.875032142Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:484 +0x51
2018-03-23T08:47:11.875035690Z created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
2018-03-23T08:47:11.875039121Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:484 +0xc7
2018-03-23T08:47:11.875042681Z
2018-03-23T08:47:11.875045965Z goroutine 50316 [runnable]:
2018-03-23T08:47:11.875049323Z github.com/ethereum/go-ethereum/consensus/ethash.makeHasher.func1(0x7f680b69f908, 0xda24c0, 0xda24c0, 0xc446059080, 0x40, 0x40)
2018-03-23T08:47:11.875052828Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/algorithm.go:101 +0xb2
2018-03-23T08:47:11.875056342Z github.com/ethereum/go-ethereum/consensus/ethash.generateCache(0x7f680a7a2008, 0x727f70, 0x727f70, 0x65, 0xc45b158b60, 0x20, 0x20)
2018-03-23T08:47:11.875059905Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/algorithm.go:188 +0x570
2018-03-23T08:47:11.875063424Z github.com/ethereum/go-ethereum/consensus/ethash.(*cache).generate.func1.1(0x7f680a7a2008, 0x727f70, 0x727f70)
2018-03-23T08:47:11.875066921Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/ethash.go:250 +0x66
2018-03-23T08:47:11.875070858Z github.com/ethereum/go-ethereum/consensus/ethash.memoryMapAndGenerate(0xc4278d03c0, 0x36, 0x1c9fdc0, 0xc422b6ee98, 0x2, 0x2, 0x0, 0x0, 0x0, 0x18359a0, ...)
2018-03-23T08:47:11.875074548Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/ethash.go:133 +0x249
2018-03-23T08:47:11.875078107Z github.com/ethereum/go-ethereum/consensus/ethash.(*cache).generate.func1()
2018-03-23T08:47:11.875081608Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/ethash.go:250 +0x664
2018-03-23T08:47:11.875085107Z sync.(*Once).Do(0xc45cdc6810, 0xc427bb5f78)
2018-03-23T08:47:11.875088494Z /usr/local/go/src/sync/once.go:44 +0xbe
2018-03-23T08:47:11.875091869Z github.com/ethereum/go-ethereum/consensus/ethash.(*cache).generate(0xc45cdc67d0, 0xc4200162c0, 0x1b, 0x3, 0xc463a7d500)
2018-03-23T08:47:11.875095468Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/ethash.go:216 +0x96
2018-03-23T08:47:11.875099004Z created by github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).cache
2018-03-23T08:47:11.875102896Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/ethash.go:506 +0x123
2018-03-23T08:47:11.875108984Z
2018-03-23T08:47:11.875112416Z goroutine 53294 [semacquire]:
2018-03-23T08:47:11.875115798Z sync.runtime_Semacquire(0xc4202346b4)
2018-03-23T08:47:11.875119158Z /usr/local/go/src/runtime/sema.go:56 +0x39
2018-03-23T08:47:11.875122544Z sync.(*RWMutex).RLock(0xc4202346a8)
2018-03-23T08:47:11.875126237Z /usr/local/go/src/sync/rwmutex.go:50 +0x49
2018-03-23T08:47:11.875129727Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).getMems(0xc420234680, 0x0, 0x0)
2018-03-23T08:47:11.875133266Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:159 +0x4c
2018-03-23T08:47:11.875136937Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).get(0xc420234680, 0x0, 0x0, 0x0, 0x0, 0xc4621ab140, 0x29, 0x30, 0x2707f95, 0x0, ...)
2018-03-23T08:47:11.875140687Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:765 +0x113
2018-03-23T08:47:11.875144251Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).Get(0xc420234680, 0xc4621ab140, 0x29, 0x30, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.875150665Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:844 +0x163
2018-03-23T08:47:11.875154195Z github.com/ethereum/go-ethereum/ethdb.(*LDBDatabase).Get(0xc42045e0d0, 0xc4621ab140, 0x29, 0x30, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.875157737Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/ethdb/database.go:121 +0xb2
2018-03-23T08:47:11.875161285Z github.com/ethereum/go-ethereum/core.GetHeaderRLP(0x7f68116f1840, 0xc42045e0d0, 0xaa97f6ab6ce1c746, 0x842b91e8d04ab703, 0xbcb28a56ac69e179, 0xa2eaecd83b581fdb, 0x2dbbaf, 0xc45937c629, 0x180fb90, 0xc46de7c820)
2018-03-23T08:47:11.875165011Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/database_util.go:163 +0x189
2018-03-23T08:47:11.875168534Z github.com/ethereum/go-ethereum/core.GetHeader(0x7f68116f1840, 0xc42045e0d0, 0xaa97f6ab6ce1c746, 0x842b91e8d04ab703, 0xbcb28a56ac69e179, 0xa2eaecd83b581fdb, 0x2dbbaf, 0x20)
2018-03-23T08:47:11.875172166Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/database_util.go:170 +0x7a
2018-03-23T08:47:11.875176081Z github.com/ethereum/go-ethereum/core.(*HeaderChain).GetHeader(0xc4203103f0, 0xaa97f6ab6ce1c746, 0x842b91e8d04ab703, 0xbcb28a56ac69e179, 0xa2eaecd83b581fdb, 0x2dbbaf, 0xc425f5d408)
2018-03-23T08:47:11.875179811Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/headerchain.go:351 +0x141
2018-03-23T08:47:11.875183420Z github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).verifyHeaderWorker(0xc420312d20, 0x1843fe0, 0xc4203103f0, 0xc442ef4600, 0xc0, 0xc0, 0xc471e869c0, 0xc0, 0xc0, 0x2, ...)
2018-03-23T08:47:11.875187100Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/consensus.go:162 +0x18a
2018-03-23T08:47:11.875190722Z github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).VerifyHeaders.func1(0xc471e88a20, 0xc471e81800, 0xc0, 0xc0, 0xc420312d20, 0x1843fe0, 0xc4203103f0, 0xc442ef4600, 0xc0, 0xc0, ...)
2018-03-23T08:47:11.875197117Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/consensus.go:116 +0xff
2018-03-23T08:47:11.875200849Z created by github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).VerifyHeaders
2018-03-23T08:47:11.875204292Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/consensus.go:114 +0x2d7
2018-03-23T08:47:11.875207848Z
2018-03-23T08:47:11.875211096Z goroutine 51227 [select]:
2018-03-23T08:47:11.875214565Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchParts(0xc4200edb00, 0x1832d20, 0xc4201b1470, 0xc4200e9620, 0xc4583c1370, 0xc4202360e0, 0xc454159eb0, 0xc454159f00, 0xc454159ef0, 0xc454159ee0, ...)
2018-03-23T08:47:11.875218292Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1004 +0x23f
2018-03-23T08:47:11.875221830Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchReceipts(0xc4200edb00, 0x2dacad, 0xc42028e798, 0x66576e8054f7f01c)
2018-03-23T08:47:11.875225401Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:957 +0x33e
2018-03-23T08:47:11.875228959Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func5(0x5fd3016700000008, 0x10086a0)
2018-03-23T08:47:11.875232472Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:465 +0x37
2018-03-23T08:47:11.875236011Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0xc425a5ca60, 0xc44c67e600, 0xc473704660)
2018-03-23T08:47:11.875239589Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:484 +0x51
2018-03-23T08:47:11.875243177Z created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
2018-03-23T08:47:11.875246906Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:484 +0xc7
2018-03-23T08:47:11.875250490Z
2018-03-23T08:47:11.875253797Z goroutine 50352 [select]:
2018-03-23T08:47:11.875257293Z github.com/ethereum/go-ethereum/consensus/ethash.generateCache.func2(0xc464dc4420, 0x1845900, 0xc45d450f00, 0xc4566468f8, 0x727f7, 0xbea54d7a4a2c3825, 0x13b37ad81b0, 0x1b91d80)
2018-03-23T08:47:11.875260992Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/algorithm.go:160 +0x12e
2018-03-23T08:47:11.875264558Z created by github.com/ethereum/go-ethereum/consensus/ethash.generateCache
2018-03-23T08:47:11.875267971Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/algorithm.go:158 +0x2f6
2018-03-23T08:47:11.875271511Z
2018-03-23T08:47:11.875274783Z goroutine 50635 [select]:
2018-03-23T08:47:11.875278119Z github.com/ethereum/go-ethereum/p2p.(*Peer).run(0xc4207f1320, 0xdb6c60, 0xc45812c680, 0x0)
2018-03-23T08:47:11.875281584Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/peer.go:204 +0x2fe
2018-03-23T08:47:11.875287386Z github.com/ethereum/go-ethereum/p2p.(*Server).runPeer(0xc4201d4280, 0xc4207f1320)
2018-03-23T08:47:11.875290956Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:894 +0x122
2018-03-23T08:47:11.875294434Z created by github.com/ethereum/go-ethereum/p2p.(*Server).run
2018-03-23T08:47:11.875297919Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:646 +0x1443
2018-03-23T08:47:11.875301430Z
2018-03-23T08:47:11.875304689Z goroutine 51328 [select]:
2018-03-23T08:47:11.875308037Z github.com/ethereum/go-ethereum/eth/downloader.(*stateSync).loop(0xc469921ab0, 0x0, 0x0)
2018-03-23T08:47:11.875311551Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:290 +0x257
2018-03-23T08:47:11.875315086Z github.com/ethereum/go-ethereum/eth/downloader.(*stateSync).run(0xc469921ab0)
2018-03-23T08:47:11.875318511Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:255 +0x2b
2018-03-23T08:47:11.875321988Z created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).runStateSync
2018-03-23T08:47:11.875325430Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/statesync.go:106 +0x134
2018-03-23T08:47:11.875328942Z
2018-03-23T08:47:11.875332197Z goroutine 51225 [select]:
2018-03-23T08:47:11.875335641Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchParts(0xc4200edb00, 0x1832d20, 0xc4201b1450, 0xc4200e9560, 0xc44f282c20, 0xc426400d90, 0xc420fdb950, 0xc420fdb990, 0xc420fdb980, 0x1006408, ...)
2018-03-23T08:47:11.875339703Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:1004 +0x23f
2018-03-23T08:47:11.875343206Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fillHeaderSkeleton(0xc4200edb00, 0x2dacad, 0xc4201e4900, 0x80, 0x8d, 0x0, 0x0, 0x0, 0xc45c5a3a80, 0xc4717cf520, ...)
2018-03-23T08:47:11.875346831Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:907 +0x37d
2018-03-23T08:47:11.875350353Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).fetchHeaders(0xc4200edb00, 0xc46c4de870, 0x2dacad, 0x5053f1, 0x0, 0x0)
2018-03-23T08:47:11.875353910Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:832 +0x130e
2018-03-23T08:47:11.875357419Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).syncWithPeer.func3(0xc400000008, 0x10086a0)
2018-03-23T08:47:11.875360887Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:463 +0x49
2018-03-23T08:47:11.875364377Z github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync.func1(0xc425a5ca60, 0xc44c67e600, 0xc45b7278c0)
2018-03-23T08:47:11.875367887Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:484 +0x51
2018-03-23T08:47:11.875371459Z created by github.com/ethereum/go-ethereum/eth/downloader.(*Downloader).spawnSync
2018-03-23T08:47:11.875379876Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/downloader/downloader.go:484 +0xc7
2018-03-23T08:47:11.875383601Z
2018-03-23T08:47:11.875386860Z goroutine 53053 [runnable]:
2018-03-23T08:47:11.875390257Z github.com/ethereum/go-ethereum/p2p/discover.(*Table).lookup(0xc4201d28c0, 0xac6f82561ac22d51, 0x7cb137a835db104b, 0xfd776dc5c81f6865, 0xf0613ecde9b77f8e, 0xc6d71626af2251e4, 0x566f9affe88e17d2, 0x9f8f01dd2a3cfffc, 0x8956809bc66c24cf, 0x1, ...)
2018-03-23T08:47:11.875394059Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:286 +0x1ff
2018-03-23T08:47:11.875397604Z github.com/ethereum/go-ethereum/p2p/discover.(*Table).Lookup(0xc4201d28c0, 0xac6f82561ac22d51, 0x7cb137a835db104b, 0xfd776dc5c81f6865, 0xf0613ecde9b77f8e, 0xc6d71626af2251e4, 0x566f9affe88e17d2, 0x9f8f01dd2a3cfffc, 0x8956809bc66c24cf, 0x0, ...)
2018-03-23T08:47:11.875401414Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/discover/table.go:279 +0x50
2018-03-23T08:47:11.875404936Z github.com/ethereum/go-ethereum/p2p.(*discoverTask).Do(0xc4635cfac0, 0xc4201d4280)
2018-03-23T08:47:11.875408836Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/dial.go:372 +0x1af
2018-03-23T08:47:11.875412401Z github.com/ethereum/go-ethereum/p2p.(*Server).run.func2.1(0x1833660, 0xc4635cfac0, 0xc4201d4280, 0xc425002ae0)
2018-03-23T08:47:11.875415945Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:570 +0x3b
2018-03-23T08:47:11.875419440Z created by github.com/ethereum/go-ethereum/p2p.(*Server).run.func2
2018-03-23T08:47:11.875423014Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/p2p/server.go:570 +0x1b3
2018-03-23T08:47:11.875426474Z
2018-03-23T08:47:11.875429748Z goroutine 53293 [semacquire]:
2018-03-23T08:47:11.875433115Z sync.runtime_Semacquire(0xc4202346b4)
2018-03-23T08:47:11.875436576Z /usr/local/go/src/runtime/sema.go:56 +0x39
2018-03-23T08:47:11.875439986Z sync.(*RWMutex).RLock(0xc4202346a8)
2018-03-23T08:47:11.875443498Z /usr/local/go/src/sync/rwmutex.go:50 +0x49
2018-03-23T08:47:11.875446905Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).getMems(0xc420234680, 0x0, 0x0)
2018-03-23T08:47:11.875450476Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:159 +0x4c
2018-03-23T08:47:11.875454148Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).get(0xc420234680, 0x0, 0x0, 0x0, 0x0, 0xc4621ab110, 0x29, 0x30, 0x2707f95, 0x0, ...)
2018-03-23T08:47:11.875457759Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:765 +0x113
2018-03-23T08:47:11.875461349Z github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).Get(0xc420234680, 0xc4621ab110, 0x29, 0x30, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.875464959Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/vendor/github.com/syndtr/goleveldb/leveldb/db.go:844 +0x163
2018-03-23T08:47:11.875471637Z github.com/ethereum/go-ethereum/ethdb.(*LDBDatabase).Get(0xc42045e0d0, 0xc4621ab110, 0x29, 0x30, 0x0, 0x0, 0x0, 0x0, 0x0)
2018-03-23T08:47:11.875475314Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/ethdb/database.go:121 +0xb2
2018-03-23T08:47:11.875478837Z github.com/ethereum/go-ethereum/core.GetHeaderRLP(0x7f68116f1840, 0xc42045e0d0, 0x9d360ca2e4dabf69, 0xb22a9570af5365fc, 0x2eb829ed560f71c8, 0xa916c1f8e2839feb, 0x2dbbb0, 0xc445909ea9, 0x180fb90, 0xc426244000)
2018-03-23T08:47:11.875482552Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/database_util.go:163 +0x189
2018-03-23T08:47:11.875486546Z github.com/ethereum/go-ethereum/core.GetHeader(0x7f68116f1840, 0xc42045e0d0, 0x9d360ca2e4dabf69, 0xb22a9570af5365fc, 0x2eb829ed560f71c8, 0xa916c1f8e2839feb, 0x2dbbb0, 0x20)
2018-03-23T08:47:11.875490222Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/database_util.go:170 +0x7a
2018-03-23T08:47:11.875493700Z github.com/ethereum/go-ethereum/core.(*HeaderChain).GetHeader(0xc4203103f0, 0x9d360ca2e4dabf69, 0xb22a9570af5365fc, 0x2eb829ed560f71c8, 0xa916c1f8e2839feb, 0x2dbbb0, 0xc46c63d408)
2018-03-23T08:47:11.875497345Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/core/headerchain.go:351 +0x141
2018-03-23T08:47:11.875500833Z github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).verifyHeaderWorker(0xc420312d20, 0x1843fe0, 0xc4203103f0, 0xc442ef4600, 0xc0, 0xc0, 0xc471e869c0, 0xc0, 0xc0, 0x3, ...)
2018-03-23T08:47:11.875504472Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/consensus.go:162 +0x18a
2018-03-23T08:47:11.875508024Z github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).VerifyHeaders.func1(0xc471e88a20, 0xc471e81800, 0xc0, 0xc0, 0xc420312d20, 0x1843fe0, 0xc4203103f0, 0xc442ef4600, 0xc0, 0xc0, ...)
2018-03-23T08:47:11.875511671Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/consensus.go:116 +0xff
2018-03-23T08:47:11.875515195Z created by github.com/ethereum/go-ethereum/consensus/ethash.(*Ethash).VerifyHeaders
2018-03-23T08:47:11.875518660Z /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/consensus/ethash/consensus.go:114 +0x2d7
2018-03-23T08:47:15.810019011Z INFO [03-23|08:47:15] Maximum peer count ETH=25 LES=0 total=25
2018-03-23T08:47:15.813359244Z INFO [03-23|08:47:15] Starting peer-to-peer node instance=Geth/v1.8.2-stable/linux-amd64/go1.9.4
2018-03-23T08:47:15.813403765Z INFO [03-23|08:47:15] Allocated cache and file handles database=/root/.ethereum/geth/chaindata cache=768 handles=1024
2018-03-23T08:47:21.032834659Z INFO [03-23|08:47:21] Initialised chain configuration config="{ChainID: 1 Homestead: 1150000 DAO: 1920000 DAOSupport: true EIP150: 2463000 EIP155: 2675000 EIP158: 2675000 Byzantium: 4370000 Constantinople: <nil> Engine: ethash}"
2018-03-23T08:47:21.032889942Z INFO [03-23|08:47:21] Disk storage enabled for ethash caches dir=/root/.ethereum/geth/ethash count=3
2018-03-23T08:47:21.032896839Z INFO [03-23|08:47:21] Disk storage enabled for ethash DAGs dir=/root/.ethash count=2
2018-03-23T08:47:21.032934585Z INFO [03-23|08:47:21] Initialising Ethereum protocol versions="[63 62]" network=1
2018-03-23T08:47:21.041455061Z INFO [03-23|08:47:21] Loaded most recent local header number=2999882 hash=c34248…adf48a td=119963881908411413293
2018-03-23T08:47:21.041494336Z INFO [03-23|08:47:21] Loaded most recent local full block number=0 hash=d4e567…cb8fa3 td=17179869184
2018-03-23T08:47:21.041500814Z INFO [03-23|08:47:21] Loaded most recent local fast block number=2995361 hash=f9fc57…1277a2 td=119496996859318936739
2018-03-23T08:47:21.047201488Z INFO [03-23|08:47:21] Loaded local transaction journal transactions=0 dropped=0
2018-03-23T08:47:21.047316609Z INFO [03-23|08:47:21] Regenerated local transaction journal transactions=0 accounts=0
2018-03-23T08:47:21.047466879Z INFO [03-23|08:47:21] Upgrading chain index type=bloombits percentage=99
2018-03-23T08:47:21.055182918Z INFO [03-23|08:47:21] Starting P2P networking
2018-03-23T08:47:21.448363979Z INFO [03-23|08:47:21] Finished upgrading chain index type=bloombits
2018-03-23T08:47:23.251858038Z INFO [03-23|08:47:23] UDP listener up self=enode://467f98b4d65dd2c87e91047df920cf42f6842ec5026cdca93c6b4dfa80338bcb93432c8b911e804dc324f90ec8a58c7b95e44d748fbd1ddfc1380e9a4c229111@[::]:30303
2018-03-23T08:47:23.254043831Z INFO [03-23|08:47:23] WebSocket endpoint opened url=ws://[::]:8546
2018-03-23T08:47:23.254215401Z INFO [03-23|08:47:23] RLPx listener up self=enode://467f98b4d65dd2c87e91047df920cf42f6842ec5026cdca93c6b4dfa80338bcb93432c8b911e804dc324f90ec8a58c7b95e44d748fbd1ddfc1380e9a4c229111@[::]:30303
2018-03-23T08:47:23.255253716Z INFO [03-23|08:47:23] IPC endpoint opened url=/root/.ethereum/geth.ipc
2018-03-23T08:47:53.256363219Z INFO [03-23|08:47:53] Block synchronisation started
2018-03-23T08:47:54.917600269Z INFO [03-23|08:47:54] Imported new block headers count=0 elapsed=5.644ms number=2995553 hash=aabecf…87c2fc ignored=192
2018-03-23T08:47:55.089727363Z INFO [03-23|08:47:55] Imported new block headers count=0 elapsed=7.148ms number=2995745 hash=83269e…897c01 ignored=192
2018-03-23T08:47:55.097301524Z INFO [03-23|08:47:55] Imported new block receipts count=0 elapsed=133.973µs number=2995363 hash=d5eec8…14420c size=0.00B ignored=2
2018-03-23T08:47:55.227679861Z INFO [03-23|08:47:55] Imported new block headers count=0 elapsed=5.100ms number=2995937 hash=b13139…3d74f3 ignored=192
2018-03-23T08:47:55.277502425Z INFO [03-23|08:47:55] Imported new block receipts count=0 elapsed=684.068µs number=2995398 hash=3151bf…11f1ee size=0.00B ignored=35
2018-03-23T08:47:55.555592356Z INFO [03-23|08:47:55] Imported new state entries count=268 elapsed=10.936µs processed=3136356 pending=4278 retry=0 duplicate=0 unexpected=0
2018-03-23T08:47:55.596098924Z INFO [03-23|08:47:55] Imported new block headers count=0 elapsed=7.858ms number=2996129 hash=317c60…4b1f8e ignored=192
2018-03-23T08:47:59.319014376Z INFO [03-23|08:47:59] Imported new block receipts count=89 elapsed=25.650ms number=2995607 hash=d42769…acb92f size=293.47kB ignored=120
2018-03-23T08:48:03.624534226Z INFO [03-23|08:48:03] Imported new block headers count=0 elapsed=7.431ms number=2996321 hash=569d64…e736c5 ignored=192
2018-03-23T08:48:03.767250140Z INFO [03-23|08:48:03] Imported new state entries count=384 elapsed=10.007µs processed=3136740 pending=10213 retry=0 duplicate=0 unexpected=0
2018-03-23T08:48:03.792492295Z INFO [03-23|08:48:03] Imported new block receipts count=219 elapsed=55.043ms number=2995826 hash=11797c…c9e98c size=830.64kB ignored=0
2018-03-23T08:48:05.888332032Z INFO [03-23|08:48:05] Imported new block headers count=0 elapsed=7.513ms number=2996513 hash=cd2d05…8069e1 ignored=192
2018-03-23T08:48:06.002482455Z INFO [03-23|08:48:06] Imported new block receipts count=201 elapsed=15.159ms number=2996027 hash=d5a977…23cbee size=570.54kB ignored=0
2018-03-23T08:48:10.861325641Z WARN [03-23|08:48:10] Node data write error err="state node 23be2e…1f310b failed with all peers (0 tries, 0 peers)"
2018-03-23T08:48:10.861366810Z WARN [03-23|08:48:10] Synchronisation failed, retrying err="block body download canceled (requested)"
2018-03-23T08:48:29.842778409Z WARN [03-23|08:48:29] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (1 tries, 1 peers)"
2018-03-23T08:48:29.842892736Z WARN [03-23|08:48:29] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (1 tries, 1 peers)"
2018-03-23T08:48:33.725051226Z WARN [03-23|08:48:33] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (1 tries, 1 peers)"
2018-03-23T08:48:33.725337612Z WARN [03-23|08:48:33] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (1 tries, 1 peers)"
2018-03-23T08:48:43.718695420Z WARN [03-23|08:48:43] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:48:43.718791927Z WARN [03-23|08:48:43] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:48:53.719332594Z WARN [03-23|08:48:53] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:48:53.722176451Z WARN [03-23|08:48:53] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:03.727181913Z WARN [03-23|08:49:03] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:03.727251778Z WARN [03-23|08:49:03] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:13.721323699Z WARN [03-23|08:49:13] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:13.721661357Z WARN [03-23|08:49:13] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:23.731557760Z WARN [03-23|08:49:23] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:23.731894941Z WARN [03-23|08:49:23] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:33.715523637Z WARN [03-23|08:49:33] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:33.715850346Z WARN [03-23|08:49:33] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:43.718693875Z WARN [03-23|08:49:43] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:43.721464645Z WARN [03-23|08:49:43] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:53.727569628Z WARN [03-23|08:49:53] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:49:53.727849213Z WARN [03-23|08:49:53] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:03.716360285Z WARN [03-23|08:50:03] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:03.718998474Z WARN [03-23|08:50:03] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:13.713366602Z WARN [03-23|08:50:13] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:13.713707024Z WARN [03-23|08:50:13] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:23.714296781Z WARN [03-23|08:50:23] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:23.714608377Z WARN [03-23|08:50:23] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:33.721508355Z WARN [03-23|08:50:33] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:33.721613076Z WARN [03-23|08:50:33] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:43.741436746Z WARN [03-23|08:50:43] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:43.741840112Z WARN [03-23|08:50:43] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:50:53.959713220Z INFO [03-23|08:50:53] Imported new block headers count=0 elapsed=3.835ms number=2996219 hash=9f3dc4…18c567 ignored=192
2018-03-23T08:50:53.975147190Z INFO [03-23|08:50:53] Imported new block headers count=0 elapsed=4.158ms number=2996411 hash=8868e8…5808ec ignored=192
2018-03-23T08:50:59.638657445Z INFO [03-23|08:50:59] Imported new block receipts count=384 elapsed=61.561ms number=2996411 hash=8868e8…5808ec size=900.52kB ignored=0
2018-03-23T08:50:59.771595945Z INFO [03-23|08:50:59] Imported new block headers count=0 elapsed=125.345ms number=2998331 hash=227080…9f4109 ignored=1920
2018-03-23T08:51:01.014657784Z INFO [03-23|08:51:01] Imported new block receipts count=226 elapsed=15.628ms number=2996637 hash=afd439…72fc88 size=595.13kB ignored=0
2018-03-23T08:51:04.902036171Z WARN [03-23|08:51:04] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:51:04.902067355Z WARN [03-23|08:51:04] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:51:15.107726559Z INFO [03-23|08:51:15] Imported new block headers count=0 elapsed=9.992ms number=2997021 hash=7e206c…91e712 ignored=384
2018-03-23T08:51:15.352044730Z INFO [03-23|08:51:15] Imported new block receipts count=2 elapsed=208.157µs number=2996639 hash=46f061…ca11b1 size=2.07kB ignored=0
2018-03-23T08:51:15.366367606Z INFO [03-23|08:51:15] Imported new block receipts count=2 elapsed=361.655µs number=2996641 hash=b5f732…354a72 size=7.05kB ignored=0
2018-03-23T08:51:21.324660492Z WARN [03-23|08:51:21] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:51:21.324779946Z WARN [03-23|08:51:21] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:51:23.778866577Z WARN [03-23|08:51:23] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:51:23.782020313Z WARN [03-23|08:51:23] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:51:33.717278414Z WARN [03-23|08:51:33] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:51:33.720130036Z WARN [03-23|08:51:33] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:51:43.720695799Z WARN [03-23|08:51:43] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:51:43.723754771Z WARN [03-23|08:51:43] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:51:53.716305961Z WARN [03-23|08:51:53] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:51:53.716372170Z WARN [03-23|08:51:53] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:03.723022292Z WARN [03-23|08:52:03] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:03.725984674Z WARN [03-23|08:52:03] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:13.724480866Z WARN [03-23|08:52:13] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:13.724856884Z WARN [03-23|08:52:13] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:23.804217319Z WARN [03-23|08:52:23] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:23.804304216Z WARN [03-23|08:52:23] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:33.714853885Z WARN [03-23|08:52:33] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:33.715253735Z WARN [03-23|08:52:33] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:43.730226736Z WARN [03-23|08:52:43] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:43.730329076Z WARN [03-23|08:52:43] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:53.719388892Z WARN [03-23|08:52:53] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:52:53.719468270Z WARN [03-23|08:52:53] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:53:03.729958630Z WARN [03-23|08:53:03] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:53:03.730272310Z WARN [03-23|08:53:03] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (2 tries, 2 peers)"
2018-03-23T08:53:14.003276967Z INFO [03-23|08:53:14] Imported new block headers count=0 elapsed=3.665ms number=2996833 hash=a12f64…cca8dd ignored=192
2018-03-23T08:53:14.614593737Z WARN [03-23|08:53:14] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:53:14.614879685Z WARN [03-23|08:53:14] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:53:25.531073973Z WARN [03-23|08:53:25] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:53:25.533901963Z WARN [03-23|08:53:25] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:53:36.764897216Z WARN [03-23|08:53:36] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:53:36.767888388Z WARN [03-23|08:53:36] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:53:43.842160725Z WARN [03-23|08:53:43] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:53:43.845312073Z WARN [03-23|08:53:43] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:53:53.999748086Z INFO [03-23|08:53:53] Imported new block headers count=0 elapsed=4.564ms number=2996833 hash=a12f64…cca8dd ignored=192
2018-03-23T08:53:54.909502073Z WARN [03-23|08:53:54] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:53:54.909801869Z WARN [03-23|08:53:54] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:03.913748906Z WARN [03-23|08:54:03] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:03.916672451Z WARN [03-23|08:54:03] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:13.878404571Z WARN [03-23|08:54:13] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:13.878579838Z WARN [03-23|08:54:13] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:24.172353452Z INFO [03-23|08:54:24] Imported new block headers count=0 elapsed=13.175ms number=2997025 hash=b8e456…1a5949 ignored=384
2018-03-23T08:54:24.364964019Z INFO [03-23|08:54:24] Imported new block receipts count=7 elapsed=464.012µs number=2996648 hash=54fea3…822f7a size=11.08kB ignored=0
2018-03-23T08:54:26.926650307Z WARN [03-23|08:54:26] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:26.926767201Z WARN [03-23|08:54:26] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:34.163410708Z WARN [03-23|08:54:34] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:34.163791135Z WARN [03-23|08:54:34] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:44.060080082Z INFO [03-23|08:54:44] Imported new block headers count=0 elapsed=7.256ms number=2997032 hash=c72426…22308e ignored=384
2018-03-23T08:54:44.307511983Z INFO [03-23|08:54:44] Imported new block receipts count=6 elapsed=683.165µs number=2996654 hash=6caf04…4f0840 size=21.30kB ignored=0
2018-03-23T08:54:45.132228568Z INFO [03-23|08:54:45] Imported new block receipts count=117 elapsed=37.842ms number=2996771 hash=39edf9…726575 size=548.56kB ignored=0
2018-03-23T08:54:45.200075693Z INFO [03-23|08:54:45] Imported new block headers count=0 elapsed=70.057ms number=2998952 hash=e1eaf0…dc1c72 ignored=1920
2018-03-23T08:54:45.220593704Z INFO [03-23|08:54:45] Imported new block headers count=0 elapsed=5.319ms number=2999144 hash=097e5e…68695b ignored=192
2018-03-23T08:54:45.238769235Z WARN [03-23|08:54:45] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:45.238918285Z INFO [03-23|08:54:45] Imported new block headers count=0 elapsed=11.034ms number=2999528 hash=eda371…5b8349 ignored=384
2018-03-23T08:54:45.246025442Z WARN [03-23|08:54:45] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:54.476595856Z INFO [03-23|08:54:54] Imported new block headers count=0 elapsed=7.022ms number=2996963 hash=f8fa4d…3f16a6 ignored=192
2018-03-23T08:54:54.557181898Z INFO [03-23|08:54:54] Imported new block headers count=0 elapsed=5.201ms number=2997155 hash=b8a722…15b1b4 ignored=192
2018-03-23T08:54:54.577392080Z INFO [03-23|08:54:54] Imported new block receipts count=2 elapsed=6.871ms number=2996773 hash=b7c163…4ad976 size=49.56kB ignored=0
2018-03-23T08:54:54.656506991Z INFO [03-23|08:54:54] Imported new block receipts count=3 elapsed=857.377µs number=2996776 hash=22878c…cce00d size=34.28kB ignored=0
2018-03-23T08:54:55.640979272Z WARN [03-23|08:54:55] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:54:55.641179331Z WARN [03-23|08:54:55] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:04.413771736Z WARN [03-23|08:55:04] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:04.414120001Z WARN [03-23|08:55:04] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:13.926848830Z INFO [03-23|08:55:13] Imported new block headers count=0 elapsed=3.622ms number=2996968 hash=3081c9…0cec90 ignored=192
2018-03-23T08:55:14.012304630Z INFO [03-23|08:55:14] Imported new block headers count=0 elapsed=3.698ms number=2997160 hash=8f29ca…a28982 ignored=192
2018-03-23T08:55:14.146705223Z INFO [03-23|08:55:14] Imported new block headers count=0 elapsed=6.362ms number=2997352 hash=6b9e1d…0a35dc ignored=192
2018-03-23T08:55:19.845650900Z WARN [03-23|08:55:19] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:19.845897234Z WARN [03-23|08:55:19] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:24.027542346Z INFO [03-23|08:55:24] Imported new block headers count=0 elapsed=10.059ms number=2997160 hash=8f29ca…a28982 ignored=384
2018-03-23T08:55:24.362700551Z INFO [03-23|08:55:24] Imported new block receipts count=5 elapsed=1.851ms number=2996781 hash=18a3ea…4498db size=83.23kB ignored=0
2018-03-23T08:55:25.333311757Z WARN [03-23|08:55:25] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:25.333516099Z WARN [03-23|08:55:25] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:34.627601680Z WARN [03-23|08:55:34] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:34.627674070Z WARN [03-23|08:55:34] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:43.887613197Z INFO [03-23|08:55:43] Imported new block headers count=0 elapsed=3.782ms number=2996973 hash=8e3b31…8b676e ignored=192
2018-03-23T08:55:43.964163832Z INFO [03-23|08:55:43] Imported new block receipts count=3 elapsed=588.057µs number=2996784 hash=093fed…91edc2 size=18.32kB ignored=0
2018-03-23T08:55:44.005924406Z INFO [03-23|08:55:44] Imported new block headers count=0 elapsed=4.710ms number=2997165 hash=dd15d0…a94781 ignored=192
2018-03-23T08:55:44.033013111Z INFO [03-23|08:55:44] Imported new block receipts count=2 elapsed=484.425µs number=2996786 hash=5370b9…b0d947 size=13.54kB ignored=0
2018-03-23T08:55:45.046573280Z WARN [03-23|08:55:45] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:45.046749367Z WARN [03-23|08:55:45] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:54.067375081Z INFO [03-23|08:55:54] Imported new block headers count=0 elapsed=4.115ms number=2996978 hash=8618dd…3537b7 ignored=192
2018-03-23T08:55:54.100558769Z INFO [03-23|08:55:54] Imported new block headers count=0 elapsed=18.261ms number=2997170 hash=82d581…be02f2 ignored=192
2018-03-23T08:55:54.224497038Z INFO [03-23|08:55:54] Imported new block receipts count=5 elapsed=809.618µs number=2996791 hash=b71473…7f17c2 size=32.31kB ignored=0
2018-03-23T08:55:57.838211367Z WARN [03-23|08:55:57] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:55:57.838380589Z WARN [03-23|08:55:57] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:56:04.521672156Z WARN [03-23|08:56:04] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:56:04.521797537Z WARN [03-23|08:56:04] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:56:13.983064566Z INFO [03-23|08:56:13] Imported new block headers count=0 elapsed=3.966ms number=2996983 hash=4b2abd…2af1ed ignored=192
2018-03-23T08:56:14.035485596Z INFO [03-23|08:56:14] Imported new block headers count=0 elapsed=4.877ms number=2997175 hash=03ae98…13ab8b ignored=192
2018-03-23T08:56:14.132057678Z INFO [03-23|08:56:14] Imported new block receipts count=2 elapsed=9.459ms number=2996793 hash=117b1a…574c05 size=26.71kB ignored=0
2018-03-23T08:56:14.132220061Z INFO [03-23|08:56:14] Imported new block receipts count=1 elapsed=121.328µs number=2996794 hash=25d464…0a7d5e size=1.56kB ignored=0
2018-03-23T08:56:14.172991313Z WARN [03-23|08:56:14] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:56:14.173091601Z WARN [03-23|08:56:14] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:56:24.424197098Z INFO [03-23|08:56:24] Imported new block headers count=0 elapsed=9.660ms number=2997178 hash=0e927e…8696d4 ignored=384
2018-03-23T08:56:24.679126325Z INFO [03-23|08:56:24] Imported new block receipts count=4 elapsed=628.582µs number=2996798 hash=526d31…c84a1f size=21.93kB ignored=0
2018-03-23T08:56:24.965919545Z WARN [03-23|08:56:24] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:56:24.966022358Z WARN [03-23|08:56:24] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (3 tries, 3 peers)"
2018-03-23T08:56:33.984746844Z WARN [03-23|08:56:33] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (4 tries, 4 peers)"
2018-03-23T08:56:33.984934150Z WARN [03-23|08:56:33] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (4 tries, 4 peers)"
2018-03-23T08:56:44.109381003Z INFO [03-23|08:56:44] Imported new block headers count=0 elapsed=7.472ms number=2997182 hash=06cf5a…9226dd ignored=384
2018-03-23T08:56:44.250830650Z INFO [03-23|08:56:44] Imported new block receipts count=2 elapsed=889.358µs number=2996800 hash=989ccf…e35d61 size=26.57kB ignored=0
2018-03-23T08:56:44.252742678Z INFO [03-23|08:56:44] Imported new block receipts count=2 elapsed=287.342µs number=2996802 hash=a6d108…c2801b size=7.57kB ignored=0
2018-03-23T08:56:44.323613624Z INFO [03-23|08:56:44] Imported new block headers count=0 elapsed=39.799ms number=2998334 hash=22c555…5fae7a ignored=1152
2018-03-23T08:56:44.402136633Z INFO [03-23|08:56:44] Imported new block headers count=0 elapsed=17.928ms number=2998910 hash=5ee3c0…eedee7 ignored=576
2018-03-23T08:56:45.936672653Z INFO [03-23|08:56:45] Imported new block receipts count=449 elapsed=41.607ms number=2997251 hash=906e10…23b1bc size=1.74mB ignored=0
2018-03-23T08:56:46.087419953Z INFO [03-23|08:56:46] Imported new block receipts count=71 elapsed=8.207ms number=2997322 hash=9cfbd9…02ce98 size=361.54kB ignored=0
2018-03-23T08:56:46.128642792Z INFO [03-23|08:56:46] Imported new block receipts count=222 elapsed=22.829ms number=2997544 hash=a90fae…9a0609 size=909.61kB ignored=0
2018-03-23T08:56:46.228589880Z WARN [03-23|08:56:46] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (4 tries, 4 peers)"
2018-03-23T08:56:46.231419471Z WARN [03-23|08:56:46] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (4 tries, 4 peers)"
2018-03-23T08:56:54.590091866Z WARN [03-23|08:56:54] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (4 tries, 4 peers)"
2018-03-23T08:56:54.601176617Z INFO [03-23|08:56:54] Imported new block headers count=0 elapsed=6.405ms number=2997736 hash=1dfd67…12b1ee ignored=192
2018-03-23T08:56:54.602387102Z WARN [03-23|08:56:54] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (4 tries, 4 peers)"
2018-03-23T08:57:04.043559276Z INFO [03-23|08:57:04] Imported new block headers count=0 elapsed=10.066ms number=2997928 hash=9220ba…ed6704 ignored=384
2018-03-23T08:57:04.144717477Z INFO [03-23|08:57:04] Imported new block receipts count=5 elapsed=414.411µs number=2997549 hash=7e24be…c8c903 size=4.86kB ignored=0
2018-03-23T08:57:04.186436425Z INFO [03-23|08:57:04] Imported new block receipts count=2 elapsed=288.445µs number=2997551 hash=4472dc…fc60c8 size=5.83kB ignored=0
2018-03-23T08:57:04.326318637Z WARN [03-23|08:57:04] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (4 tries, 4 peers)"
2018-03-23T08:57:04.328462505Z WARN [03-23|08:57:04] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (4 tries, 4 peers)"
2018-03-23T08:57:14.855004722Z INFO [03-23|08:57:14] Imported new block headers count=0 elapsed=3.724ms number=2997743 hash=c91935…d1a5f4 ignored=192
2018-03-23T08:57:14.975804845Z INFO [03-23|08:57:14] Imported new block headers count=0 elapsed=5.012ms number=2997935 hash=7fac1c…21f920 ignored=192
2018-03-23T08:57:15.216660060Z INFO [03-23|08:57:15] Imported new block headers count=0 elapsed=15.425ms number=2998319 hash=450522…9b7cbd ignored=384
2018-03-23T08:57:15.378393319Z WARN [03-23|08:57:15] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (5 tries, 5 peers)"
2018-03-23T08:57:15.473274759Z INFO [03-23|08:57:15] Imported new block headers count=0 elapsed=67.742ms number=3000047 hash=ecd395…9d8a37 ignored=1728
2018-03-23T08:57:15.482860829Z WARN [03-23|08:57:15] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (5 tries, 5 peers)"
2018-03-23T08:57:19.602089615Z INFO [03-23|08:57:19] Imported new block headers count=0 elapsed=5.402ms number=2997743 hash=c91935…d1a5f4 ignored=192
2018-03-23T08:57:19.916238599Z INFO [03-23|08:57:19] Imported new block receipts count=7 elapsed=1.266ms number=2997558 hash=41ea40…0fa70d size=14.99kB ignored=0
2018-03-23T08:57:19.939042348Z INFO [03-23|08:57:19] Imported new block headers count=0 elapsed=15.762ms number=2998319 hash=450522…9b7cbd ignored=576
2018-03-23T08:57:24.934173995Z WARN [03-23|08:57:24] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (5 tries, 5 peers)"
2018-03-23T08:57:25.081654135Z INFO [03-23|08:57:25] Imported new block headers count=0 elapsed=110.828ms number=3000367 hash=2874d8…2c30ed ignored=2048
2018-03-23T08:57:25.353860826Z INFO [03-23|08:57:25] Imported new block receipts count=761 elapsed=426.244ms number=2998319 hash=450522…9b7cbd size=4.95mB ignored=0
2018-03-23T08:57:25.520569276Z INFO [03-23|08:57:25] Imported new block headers count=485 elapsed=371.505ms number=3002415 hash=52a122…6bc8df ignored=1563
2018-03-23T08:57:25.551729477Z WARN [03-23|08:57:25] Rolled back headers count=485 header=3002415->3001930 fast=2998319->2998319 block=0->0
2018-03-23T08:57:25.552382840Z WARN [03-23|08:57:25] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (5 tries, 5 peers)"
2018-03-23T08:57:28.030224723Z INFO [03-23|08:57:28] Imported new block headers count=0 elapsed=6.301ms number=2998511 hash=10ee4e…493d00 ignored=192
2018-03-23T08:57:28.141148367Z INFO [03-23|08:57:28] Imported new block headers count=0 elapsed=6.745ms number=2998703 hash=d40b01…b2785e ignored=192
2018-03-23T08:57:28.276364231Z INFO [03-23|08:57:28] Imported new block receipts count=7 elapsed=457.036µs number=2998326 hash=248bd0…9edc75 size=8.69kB ignored=0
2018-03-23T08:57:30.394619689Z WARN [03-23|08:57:30] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (5 tries, 5 peers)"
2018-03-23T08:57:30.395907297Z WARN [03-23|08:57:30] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (5 tries, 5 peers)"
2018-03-23T08:57:32.352788234Z INFO [03-23|08:57:32] Imported new block headers count=0 elapsed=6.519ms number=2998518 hash=e4c9c4…5fd568 ignored=192
2018-03-23T08:57:33.002415337Z INFO [03-23|08:57:33] Imported new block receipts count=3 elapsed=754.58µs number=2998329 hash=5650df…e535c9 size=29.10kB ignored=0
2018-03-23T08:57:33.530601422Z INFO [03-23|08:57:33] Imported new block receipts count=189 elapsed=26.823ms number=2998518 hash=e4c9c4…5fd568 size=585.40kB ignored=0
2018-03-23T08:57:33.593773543Z WARN [03-23|08:57:33] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (5 tries, 5 peers)"
2018-03-23T08:57:33.594711013Z INFO [03-23|08:57:33] Imported new block headers count=0 elapsed=70.194ms number=3000566 hash=177f06…3a5a10 ignored=2048
2018-03-23T08:57:33.606224755Z WARN [03-23|08:57:33] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (5 tries, 5 peers)"
2018-03-23T08:57:36.098072262Z INFO [03-23|08:57:36] Imported new block headers count=0 elapsed=5.216ms number=2998710 hash=316d8a…d79f80 ignored=192
2018-03-23T08:57:36.150104789Z INFO [03-23|08:57:36] Imported new block headers count=0 elapsed=5.383ms number=2998902 hash=569821…349e7d ignored=192
2018-03-23T08:57:36.156180744Z WARN [03-23|08:57:36] Node data write error err="state node 4ef0e6…525bc4 failed with all peers (5 tries, 5 peers)"
2018-03-23T08:57:36.156268512Z WARN [03-23|08:57:36] Synchronisation failed, retrying err="state node 4ef0e6…525bc4 failed with all peers (5 tries, 5 peers)"
2018-03-23T08:57:36.681773726Z WARN [03-23|08:57:36] Synchronisation failed, dropping peer peer=a6bef54e4cb56bbb err="action from bad peer ignored"
2018-03-23T08:57:43.644424463Z INFO [03-23|08:57:43] Imported new block headers count=0 elapsed=4.954ms number=2998710 hash=316d8a…d79f80 ignored=192
2018-03-23T08:57:43.725809402Z INFO [03-23|08:57:43] Imported new block receipts count=2 elapsed=243.137µs number=2998520 hash=8cf526…a6ebfa size=2.56kB ignored=0
2018-03-23T08:57:44.336399843Z INFO [03-23|08:57:44] Imported new block receipts count=142 elapsed=35.509ms number=2998662 hash=c9f4f6…b0223b size=423.80kB ignored=0
2018-03-23T08:57:44.420128592Z INFO [03-23|08:57:44] Imported new block headers count=0 elapsed=79.288ms number=3000758 hash=8fb6bb…d9c72d ignored=2048
2018-03-23T08:57:44.459251522Z INFO [03-23|08:57:44] Imported new block headers count=0 elapsed=10.810ms number=3001014 hash=7af6d5…bdf2cd ignored=256
2018-03-23T08:57:44.474478262Z INFO [03-23|08:57:44] Imported new block headers count=0 elapsed=10.908ms number=3001206 hash=04f763…57ecfc ignored=192
2018-03-23T08:57:44.576312942Z INFO [03-23|08:57:44] Imported new block headers count=0 elapsed=10.753ms number=3001590 hash=91bbc2…57dd07 ignored=384
2018-03-23T08:57:44.734951868Z WARN [03-23|08:57:44] Node data write error err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:57:44.736137051Z WARN [03-23|08:57:44] Synchronisation failed, retrying err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:57:53.725476000Z INFO [03-23|08:57:53] Imported new block headers count=0 elapsed=7.601ms number=2998854 hash=4dc9f3…d1cb1b ignored=192
2018-03-23T08:57:55.089747304Z WARN [03-23|08:57:55] Node data write error err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:57:55.089932628Z WARN [03-23|08:57:55] Synchronisation failed, retrying err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:04.578736061Z WARN [03-23|08:58:04] Node data write error err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:04.578799950Z WARN [03-23|08:58:04] Synchronisation failed, retrying err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:13.932759151Z INFO [03-23|08:58:13] Imported new block headers count=0 elapsed=4.845ms number=2998854 hash=4dc9f3…d1cb1b ignored=192
2018-03-23T08:58:15.015819561Z INFO [03-23|08:58:15] Imported new block receipts count=3 elapsed=263.627µs number=2998665 hash=2f85e3…359404 size=2.27kB ignored=0
2018-03-23T08:58:15.056406851Z INFO [03-23|08:58:15] Imported new block receipts count=189 elapsed=35.623ms number=2998854 hash=4dc9f3…d1cb1b size=656.87kB ignored=0
2018-03-23T08:58:15.124570060Z INFO [03-23|08:58:15] Imported new block headers count=0 elapsed=68.954ms number=3000902 hash=9ef14b…533915 ignored=2048
2018-03-23T08:58:15.183305720Z WARN [03-23|08:58:15] Node data write error err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:15.301196762Z INFO [03-23|08:58:15] Imported new block headers count=535 elapsed=140.698ms number=3002950 hash=6c3468…0492d5 ignored=1513
2018-03-23T08:58:15.330845671Z WARN [03-23|08:58:15] Rolled back headers count=535 header=3002950->3002415 fast=2998854->2998854 block=0->0
2018-03-23T08:58:15.330865434Z WARN [03-23|08:58:15] Synchronisation failed, retrying err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:23.646073636Z INFO [03-23|08:58:23] Imported new block headers count=0 elapsed=5.038ms number=2999046 hash=32f930…fac81f ignored=192
2018-03-23T08:58:23.871591655Z INFO [03-23|08:58:23] Imported new block receipts count=5 elapsed=794.373µs number=2998859 hash=6a1e1b…e34e59 size=20.92kB ignored=0
2018-03-23T08:58:29.939885097Z INFO [03-23|08:58:29] Imported new block receipts count=187 elapsed=24.339ms number=2999046 hash=32f930…fac81f size=700.39kB ignored=0
2018-03-23T08:58:30.025026843Z INFO [03-23|08:58:30] Imported new block headers count=0 elapsed=66.217ms number=3001094 hash=4c25e8…bc461e ignored=2048
2018-03-23T08:58:30.142040026Z INFO [03-23|08:58:30] Imported new block headers count=192 elapsed=86.451ms number=3003142 hash=02c9a2…eb8499 ignored=1856
2018-03-23T08:58:30.504591303Z INFO [03-23|08:58:30] Imported new block receipts count=106 elapsed=16.307ms number=2999152 hash=37a412…40127e size=325.40kB ignored=0
2018-03-23T08:58:30.523237671Z INFO [03-23|08:58:30] Imported new block headers count=2048 elapsed=341.262ms number=3005190 hash=2f6a8a…3523a9 ignored=0
2018-03-23T08:58:30.815576156Z INFO [03-23|08:58:30] Imported new block receipts count=76 elapsed=10.522ms number=2999228 hash=a3693b…1195bf size=183.41kB ignored=0
2018-03-23T08:58:30.924303047Z INFO [03-23|08:58:30] Imported new block headers count=2048 elapsed=363.932ms number=3007238 hash=622f78…981c5c ignored=0
2018-03-23T08:58:31.115262607Z INFO [03-23|08:58:31] Imported new block receipts count=319 elapsed=54.379ms number=2999547 hash=77abd9…3af2ff size=871.96kB ignored=0
2018-03-23T08:58:31.321028106Z INFO [03-23|08:58:31] Imported new block headers count=2048 elapsed=354.759ms number=3009286 hash=6839df…25db11 ignored=0
2018-03-23T08:58:31.599147157Z INFO [03-23|08:58:31] Imported new block receipts count=287 elapsed=41.327ms number=2999834 hash=716ffb…a6f310 size=1.08mB ignored=0
2018-03-23T08:58:31.720922479Z INFO [03-23|08:58:31] Imported new block headers count=2048 elapsed=360.562ms number=3011334 hash=44b471…b14916 ignored=0
2018-03-23T08:58:31.825917935Z WARN [03-23|08:58:31] Node data write error err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:32.094233124Z INFO [03-23|08:58:32] Imported new block headers count=2048 elapsed=333.462ms number=3013382 hash=625ffa…c511e5 ignored=0
2018-03-23T08:58:32.185667811Z WARN [03-23|08:58:32] Rolled back headers count=2048 header=3013382->3011334 fast=2999834->2999834 block=0->0
2018-03-23T08:58:32.185750569Z WARN [03-23|08:58:32] Synchronisation failed, retrying err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:33.824233312Z INFO [03-23|08:58:33] Imported new block headers count=0 elapsed=5.210ms number=3000026 hash=181d53…c6fd3e ignored=192
2018-03-23T08:58:33.873305043Z INFO [03-23|08:58:33] Imported new block headers count=0 elapsed=10.671ms number=3000410 hash=950115…d3da84 ignored=384
2018-03-23T08:58:34.318227341Z WARN [03-23|08:58:34] Node data write error err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:34.319287351Z WARN [03-23|08:58:34] Synchronisation failed, retrying err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:45.691980648Z WARN [03-23|08:58:45] Node data write error err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:45.692116757Z WARN [03-23|08:58:45] Synchronisation failed, retrying err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:53.681344191Z INFO [03-23|08:58:53] Imported new block headers count=0 elapsed=5.163ms number=3000026 hash=181d53…c6fd3e ignored=192
2018-03-23T08:58:53.991278148Z WARN [03-23|08:58:53] Node data write error err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
2018-03-23T08:58:53.996931870Z WARN [03-23|08:58:53] Synchronisation failed, retrying err="state node 80e90b…69c5ae failed with all peers (4 tries, 4 peers)"
sulliwane
All 24 comments
Also receiving this issue. Geth slowly eats up all available ram on the system until it crashes.
Edit:
Ended up increasing the server ram to 8gb, things are running okay.
TediPapajorgji
on 29 Mar 2018
Having the same issue, built from the latest 1.8.3 source, Linux, RAM 2 GB. Sync mode fast, Ropsten network.
Is it possible to limit geth's RAM usage instead of increasing the server RAM?
UPD: adding a 8 GB swap file didn't help. Same behaviour: eats up all available memory and dies.
sapph1re
on 30 Mar 2018
Also, has this problem appeared only in 1.8.2? Will reverting to 1.8.1 or 1.8.0 help?
sapph1re
on 30 Mar 2018
Hey Sapph1re, I dont think there's anyway to limit geth's ram usage at the current moment.
Also not too sure if reverting to an older version will help. Although this being said, after upgrading to 8gb of ram my geth has not crashed and it's ram usage seems to have stabilized around 4gb.
I've included the systemd service file that i'm using to launch geth, it may be of use to you:
[Unit]
Description=The Ethereum Blockchain Command Line Interface
Documentation=man:geth(1)
After=network.target
[Service]
User=ethereum
ExecStart=/usr/bin/geth --syncmode "fast" --cache 1024 --rpcaddr "0.0.0.0" --rpcport 9551 --rpc --rpcapi personal,eth,ssh,web3 --rpccorsdomain "52.224.54.206" 2>&1 >> /home/ethereum/geth.l$
Restart=always
[Install]
WantedBy=default.target
Server specifications:
Ubuntu 16.04.4 x64
8 GB Memory
160 GB SSD Disk
4 vCPU's
digitalocean.com droplet
TediPapajorgji
on 30 Mar 2018
Setting the --cache flag is supposed to do it but there is either a mem-leak or it's not respecting it. We've also noticed this both in fast-sync mode and proper full-sync mode on mainnet and ropsten testnet.
In general we've noticed that when syncing Geth will eat up as much memory as there is available on the machine. I don't think this an issue with just v1.8.2. We've seen it all the back to v1.7.x.
Once in sync the memory and CPU profile of Geth drastically decreases but it spikes every time a block is verified (for obvious reasons). So you need to give it quite a bit more than your --cache value. In, general we've been seeing things stabilizing at whatever your cache value is + 3GB.
ekryski
on 13 Apr 2018
I am having an "out of memory" issue with Geth 1.8.9 that is not triggered by fast sync, but just by running Geth. Whenever I start Geth, if I let it run, after a while, it will crash with an "out of memory" error message. My Ubutun Xenial server is configured with 8GB of RAM and 32GB of swap. The command line to start Geth is the following: go-ethereum/build/bin/geth --nodiscover --cache=2048 --rpc --rpcaddr 0.0.0.0 --rpccorsdomain "*" --rpcapi "db,eth,net,web3,personal,debug,txpool" js ./ethereum/mine_pending.js
i used to run Geth 1.6.x with the same command and never had an issue with it running out of memory. I have actually tripled the swap space on this server to accommodate the "out of memory" issue and it is still there.
dbblackdiamond
on 31 May 2018
Yeah. Running more nodes, this is becoming a real problem. Both in ropsten testnet and mainnet. There is a memory leak somewhere. Geth continually eats up memory until it runs out of machine memory.
Seeing this in v1.8.2 and later versions. But we haven't been able to upgrade passed v1.8.2 due to https://github.com/ethereum/go-ethereum/issues/16846
We're running on Kubernetes so this is particularly painful because our Geth nodes continually get restarted.
ekryski
on 16 Jun 2018
I should probably have mentioned that I'm running 3 geth dockers on one single machine (Mainnet, Ropsten, Rinkeby).
sulliwane
on 16 Jun 2018
I notice that it crash after few hours only once I activate --rpcapi "admin,db,eth,miner,net,web3,personal,txpool"
Without that argument, geth is running fine on ropsten.
The failing setup:
cat docker-compose.yml
version: "3"
services:
geth_testnet:
container_name: geth_testnet
image: ethereum/client-go:v1.8.16
command: '--testnet --syncmode "fast" --rpc --rpcaddr 0.0.0.0 --rpccorsdomain "*" --rpcvhosts "*" --rpcapi "admin,db,eth,miner,net,web3,personal,txpool"'
ports:
- "8545:8545"
- "30303:30303"
- "30303:30303/udp"
volumes:
- testnet_data:/root/.ethereum
volumes:
testnet_data:
Hope that's help!
gregbkr
on 28 Sep 2018
I notice that it crash after few hours only once I activate
--rpcapi "admin,db,eth,miner,net,web3,personal,txpool"
Without that argument, geth is running fine on ropsten.The failing setup:
cat docker-compose.yml version: "3" services: geth_testnet: container_name: geth_testnet image: ethereum/client-go:v1.8.16 command: '--testnet --syncmode "fast" --rpc --rpcaddr 0.0.0.0 --rpccorsdomain "*" --rpcvhosts "*" --rpcapi "admin,db,eth,miner,net,web3,personal,txpool"' ports: - "8545:8545" - "30303:30303" - "30303:30303/udp" volumes: - testnet_data:/root/.ethereum volumes: testnet_data:Hope that's help!
Ah thats interesting! I upgraded my node to 16gb of memory and geth has yet to crash on me anymore. The memory limit flag doesn't actually limit geth's memory usage it seems.
TediPapajorgji
on 28 Sep 2018
same issue.
when i change from centos7.4 to ubuntu16.04 and change the rpcport from 8581 to 8681. it seems to be stoped! i think the rpcport may be the main case.
here is my specifications:
Server
Ubuntu 16.04.4 x64
4 GB Memory
40 GB HDD Disk
2 vCPU's
Geth
Version: 1.8.16-stable
Architecture: amd64
Go Version: go1.11.1
eliceaas
on 17 Oct 2018
you should be get a rpc attack.
At 2018-10-17 10:33:05, "eliceaas" notifications@github.com wrote:
same issue.
when i change from centos7.4 to ubuntu16.04 and change the rpcport from 8581 to 8681. it seems to be stoped! i think the rpcport may be the main case.
here is my specifications:
Server
Ubuntu 16.04.4 x64
4 GB Memory
40 GB HDD Disk
2 vCPU's
Geth
Version: 1.8.16-stable
Architecture: amd64
Go Version: go1.11.1
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
matingwei
on 17 Oct 2018
hi, guys, I also encountered this problem. But when I removed the "--rpc" config ,geth works well. I think the RPC service is the main problem, and hope someone can fix this.
hashfury42
on 17 Oct 2018
The main problem is most likely attackers on the internet discovering the rpc service and doing bruteforce password guessing against personal.unlock. Luckily, decrypting keystore is very memory intensive.
Note: this doesn't seem to be the case for the original reporter, but very likely for those of you that experience problems only when rpc is enabled.
holiman
on 17 Oct 2018
The main problem is most likely attackers on the internet discovering the rpc service and doing bruteforce password guessing against personal.unlock. Luckily, decrypting keystore is very memory intensive.
Note: this doesn't seem to be the case for the original reporter, but very likely for those of you that experience problems only when rpc is enabled.
hi, holiman
I set up a firewall to proctect the RPC port, and this problem has disappeared.
Thank you very much, and I hope this will help someone else who encountered this problem. 👍
hashfury42
on 18 Oct 2018
@hashfury42 could you please share how did you setup your firewall on port, did you allow only particular ip to have access to that port?
NIVJAIN
on 22 Oct 2018
@hashfury42 could you please share how did you setup your firewall on port, did you allow only particular ip to have access to that port?
NIVJAIN
on 22 Oct 2018
@hashfury42 could you please share how did you setup your firewall on port, did you allow only particular ip to have access to that port?
Yes, only particular ip have access to that port, if you use Ubuntu you can read this https://help.ubuntu.com/community/UFW
hashfury42
on 22 Oct 2018
@holiman
thanks
in my case this method helped me.
I created a solution on: https://stackoverflow.com/questions/53206228/ethereum-geth-out-of-memory/
I hope it will be useful
Ninjamannn
on 8 Nov 2018
Hi Alex,
Greetings from Singapore and thanks alot for the information. I will try
this out by tomorrow morning.
one more question, if i want to enable for two ip addresses, do i need to
repeat the iptable commands twice with different ip addresses.
regards
Jain
On Thu, Nov 8, 2018 at 7:29 PM Alex notifications@github.com wrote:
@holiman https://github.com/holiman
thanks
in my case this method helped me.
I created a solution on:
https://stackoverflow.com/questions/53206228/ethereum-geth-out-of-memory/
http://url
I hope it will be useful—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ethereum/go-ethereum/issues/16377#issuecomment-436963379,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AfcavuLJMbHbAj4CaFWtaTF3fdr7wF4Pks5utBWigaJpZM4S4X8s
.
NIVJAIN
on 13 Nov 2018
@NIVJAIN
Hi Jain!
repeat the iptable commands twice with different ip
if I understand correctly, yes. However, I recommend reading about linux iptables. Your main task is to have strictly defined addresses access to your node on the port.
Provided access to node 2 ip addresses do not forget to specify this parameter: -- rpccorsdomain
--rpccorsdomain value Comma separated *list of domains from which to accept cross origin requests (browser enforced)*
please write about your results...
Ninjamannn
on 13 Nov 2018
I'm not sure if it's same problem or not.
I used to be troubled in same issue geth's "out of memory".
so I expanded AWS Server spec and the problem was gone.
but new issue came, CPU and MEM high usage.
The issue is it slowly increases to cpu and mem limit.
it cause AWS Server paralysis
Server
AWS t3.medium(64bit - ubuntu 16.04, 2core, 4GB Ram, 8GB Swap memory)
version : 1.8.1-stable, 1.8.16-stable
fast and light syncmode both.
geth --rinkeby --rpc --rpcaddr "0.0.0.0" --rpcvhosts=* --rpcport "8545" --rpccorsdomain "neojuneELB-1439772252.ap-northeast-2.elb.amazonaws.com" --rpcapi "eth,net,web3,personal,admin" --syncmode "light" --cache "64"
hapsody
on 8 Jan 2019
It happened with me right now on my private ethereum network. Is there any progress on this?
Geth command:-
geth --rpc --rpcport "8545" --rpcaddr "127.0.0.1" --rpccorsdomain "*" --ws --wsorigins "*" --wsaddr "127.0.0.1" --wsport "8546" --rpcapi "web3,personal,eth,net" --wsapi personal,web3,eth,net,db --cache 2048 console
Log:-
fatal error: runtime: out of memory
runtime stack:
runtime.throw(0x11ecc01, 0x16)
/build/ethereum-CazSBy/.go/src/runtime/panic.go:774 +0x72
runtime.sysMap(0xc194000000, 0x10000000, 0x20db3f8)
/build/ethereum-CazSBy/.go/src/runtime/mem_linux.go:169 +0xc5
runtime.(*mheap).sysAlloc(0x20a9840, 0x10000000, 0x0, 0x882b79)
/build/ethereum-CazSBy/.go/src/runtime/malloc.go:701 +0x1cd
runtime.(*mheap).grow(0x20a9840, 0x8000, 0x1ffffffff)
/build/ethereum-CazSBy/.go/src/runtime/mheap.go:1255 +0xa3
runtime.(*mheap).allocSpanLocked(0x20a9840, 0x8000, 0x20db408, 0x181b730)
/build/ethereum-CazSBy/.go/src/runtime/mheap.go:1170 +0x266
runtime.(*mheap).alloc_m(0x20a9840, 0x8000, 0x7f0084680101, 0x7f00846879a0)
/build/ethereum-CazSBy/.go/src/runtime/mheap.go:1022 +0xc2
runtime.(*mheap).alloc.func1()
/build/ethereum-CazSBy/.go/src/runtime/mheap.go:1093 +0x4c
runtime.(*mheap).alloc(0x20a9840, 0x8000, 0x7f0084010101, 0x7f007d7b4eb0)
/build/ethereum-CazSBy/.go/src/runtime/mheap.go:1092 +0x8a
runtime.largeAlloc(0x10000000, 0x7f007d7b0101, 0x46fabe)
princesinha19
on 21 Feb 2020
This is a very old ticket, and we've worked on, and fixed many memory issues over time. I'm closing this, it's better if new tickets are opened relating to the more recent versions
holiman
on 10 Sep 2020
Related issues
leonzhao
·
3Comments
phpsamsb
·
3Comments
cheershendtco
·
3Comments
VenusHu
·
3Comments
aakilfernandes
·
3Comments
Most helpful comment
Hey Sapph1re, I dont think there's anyway to limit geth's ram usage at the current moment.
Also not too sure if reverting to an older version will help. Although this being said, after upgrading to 8gb of ram my geth has not crashed and it's ram usage seems to have stabilized around 4gb.
I've included the systemd service file that i'm using to launch geth, it may be of use to you:
Server specifications:
Ubuntu 16.04.4 x64
8 GB Memory
160 GB SSD Disk
4 vCPU's
digitalocean.com droplet