Go-ethereum: eth.HeaderByNumber randomly causing geth to panic

[shoenseiwaso]

System information

Geth version: v1.9.9
OS & Version: Linux ubuntu-18.04

Expected behaviour

Geth does not panic

Actual behaviour

Geth panics seemingly randomly when handling an RPC call that involves computing the gas price for a transaction. The crash itself happens in eth.HeaderByNumber() on line 70 here:
https://github.com/ethereum/go-ethereum/blob/v1.9.9/eth/api_backend.go#L68-L71

Steps to reproduce the behaviour

We haven't been able to reliably recreate the issue - sometimes it fails, sometimes not. We have seen about 20 occurrences both in our CI and on local developer machines, all over the past two days. We haven't been able to find a common link as to why we're seeing the crashes now.

Will keep this issue updated as/if we learn more.

Backtrace

ERROR[01-08|06:13:30.145] RPC method eth_gasPrice crashed: runtime error: invalid memory address or nil pointer dereference
goroutine 1494 [running]:
github.com/ethereum/go-ethereum/rpc.(*callback).call.func1(0xc018eedaf0, 0xc, 0xc0192eed50)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/service.go:200 +0xb5
panic(0xff5140, 0x1b64bd0)
    /usr/local/go/src/runtime/panic.go:679 +0x1b2
github.com/ethereum/go-ethereum/eth.(*EthAPIBackend).HeaderByNumber(0x0, 0x13fdc40, 0xc000ec25c0, 0xffffffffffffffff, 0x104f9e0, 0xc0192ee890, 0xc00004def0)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/api_backend.go:70 +0x3f
github.com/ethereum/go-ethereum/eth/gasprice.(*Oracle).SuggestPrice(0xc0000dc980, 0x13fdc40, 0xc000ec25c0, 0x0, 0x0, 0x0)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/gasprice/gasprice.go:83 +0xe1
github.com/ethereum/go-ethereum/eth.(*EthAPIBackend).SuggestPrice(0xc00042a2e0, 0x13fdc40, 0xc000ec25c0, 0x1c9c040, 0x7f80ffd3b008, 0x0)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/api_backend.go:275 +0x43
github.com/ethereum/go-ethereum/internal/ethapi.(*PublicEthereumAPI).GasPrice(0xc0002b9970, 0x13fdc40, 0xc000ec25c0, 0x0, 0x0, 0x0)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/internal/ethapi/api.go:67 +0x4a
reflect.Value.call(0xc018aa9080, 0xc000010470, 0x13, 0x115e110, 0x4, 0xc000e87d40, 0x2, 0x2, 0xc0192eebc8, 0x4601ec, ...)
    /usr/local/go/src/reflect/value.go:460 +0x5f6
reflect.Value.Call(0xc018aa9080, 0xc000010470, 0x13, 0xc000e87d40, 0x2, 0x2, 0x0, 0x0, 0xc0192eec88)
    /usr/local/go/src/reflect/value.go:321 +0xb4
github.com/ethereum/go-ethereum/rpc.(*callback).call(0xc018b4a120, 0x13fdc40, 0xc000ec25c0, 0xc018eedaf0, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/service.go:206 +0x2b9
github.com/ethereum/go-ethereum/rpc.(*handler).runMethod(0xc0005f7950, 0x13fdc40, 0xc000ec25c0, 0xc000ebe5b0, 0xc018b4a120, 0x0, 0x0, 0x0, 0x0)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:369 +0x8a
github.com/ethereum/go-ethereum/rpc.(*handler).handleCall(0xc0005f7950, 0xc000e87ce0, 0xc000ebe5b0, 0xc018bea6d0)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:331 +0x22c
github.com/ethereum/go-ethereum/rpc.(*handler).handleCallMsg(0xc0005f7950, 0xc000e87ce0, 0xc000ebe5b0, 0x13e9401)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:298 +0x1f3
github.com/ethereum/go-ethereum/rpc.(*handler).handleMsg.func1(0xc000e87ce0)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:139 +0x46
github.com/ethereum/go-ethereum/rpc.(*handler).startCallProc.func1(0xc0005f7950, 0xc000f24440)
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:226 +0x119
created by github.com/ethereum/go-ethereum/rpc.(*handler).startCallProc
    /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:222 +0x66

WARN [01-08|06:13:30.145] Served eth_gasPrice                      reqid=873 t=202.502µs err="method handler crashed"

[geomad]

I have the same issue. It happens everytime I try to call web3.eth.getGasPrice() . This started happening after yesterday's updates (now running Geth/v1.9.10-unstable-0218d700-20200108)

Backtrace
ERROR[01-08|10:19:02.697] RPC method eth_gasPrice crashed: runtime error: invalid memory address or nil pointer dereference goroutine 772539 [running]: github.com/ethereum/go-ethereum/rpc.(*callback).call.func1(0x4c565160, 0xc, 0x83acea8) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/service.go:200 +0x74 panic(0xa72718, 0x145b048) /usr/local/go/src/runtime/panic.go:679 +0x194 github.com/ethereum/go-ethereum/eth.(*EthAPIBackend).HeaderByNumber(0x0, 0xde8298, 0x453bf180, 0xffffffff, 0xffffffff, 0x300f8a0, 0xde8298, 0x453bea00) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/api_backend.go:70 +0x5c github.com/ethereum/go-ethereum/eth/gasprice.(*Oracle).SuggestPrice(0x31ea300, 0xde8298, 0x453bf180, 0x0, 0x0, 0x0) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/gasprice/gasprice.go:83 +0xc4 github.com/ethereum/go-ethereum/eth.(*EthAPIBackend).SuggestPrice(0x300f8a0, 0xde8298, 0x453bf180, 0xf77bf008, 0x0, 0x36008) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/eth/api_backend.go:275 +0x30 github.com/ethereum/go-ethereum/internal/ethapi.(*PublicEthereumAPI).GasPrice(0x41324b8, 0xde8298, 0x453bf180, 0x0, 0x0, 0x0) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/internal/ethapi/api.go:67 +0x38 reflect.Value.call(0x5abdf40, 0x3f48970, 0x13, 0xb4c03b, 0x4, 0x453bf1e0, 0x2, 0x2, 0xb48df8, 0x4095d4, ...) /usr/local/go/src/reflect/value.go:460 +0x49c reflect.Value.Call(0x5abdf40, 0x3f48970, 0x13, 0x453bf1e0, 0x2, 0x2, 0x0, 0x0, 0x0) /usr/local/go/src/reflect/value.go:321 +0x78 github.com/ethereum/go-ethereum/rpc.(*callback).call(0x5ba5320, 0xde8298, 0x453bf180, 0x4c565160, 0xc, 0x159485c, 0x0, 0x0, 0x0, 0x0, ...) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/service.go:206 +0x22c github.com/ethereum/go-ethereum/rpc.(*handler).runMethod(0x2a4413b0, 0xde8298, 0x453bf180, 0x417edf80, 0x5ba5320, 0x159485c, 0x0, 0x0, 0x0) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:369 +0x58 github.com/ethereum/go-ethereum/rpc.(*handler).handleCall(0x2a4413b0, 0x453bf1a0, 0x417edf80, 0x7c6) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:331 +0x1e4 github.com/ethereum/go-ethereum/rpc.(*handler).handleCallMsg(0x2a4413b0, 0x453bf1a0, 0x417edf80, 0xa64a78) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:298 +0x17c github.com/ethereum/go-ethereum/rpc.(*handler).handleMsg.func1(0x453bf1a0) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:139 +0x30 github.com/ethereum/go-ethereum/rpc.(*handler).startCallProc.func1(0x2a4413b0, 0x21f2c590) /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:226 +0xd4 created by github.com/ethereum/go-ethereum/rpc.(*handler).startCallProc /go-ethereum/build/_workspace/src/github.com/ethereum/go-ethereum/rpc/handler.go:222 +0x50

[karalabe]

Fixed on master, a PR accidentally left a field nil.

[shoenseiwaso]

Great, thank you @karalabe and @rjl493456442!