Web3.py: Unhandled exception deploying contracts with Infura

[fubuloubu]

• Version: 4.8.1
• Python: 3.6.6
• OS: linux
• pip freeze output:

attrdict==2.0.0
backcall==0.1.0
certifi==2018.10.15
chardet==3.0.4
cytoolz==0.9.0.1
decorator==4.3.0
eth-abi==1.2.2
eth-account==0.3.0
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.2.0b3
eth-rlp==0.1.2
eth-typing==1.3.0
eth-utils==1.2.2
hexbytes==0.1.0
idna==2.7
ipython==7.1.1
ipython-genutils==0.2.0
jedi==0.13.1
lru-dict==1.1.6
parsimonious==0.8.1
parso==0.3.1
pexpect==4.6.0
pickleshare==0.7.5
prompt-toolkit==2.0.7
ptyprocess==0.6.0
pycryptodome==3.7.0
Pygments==2.2.0
requests==2.20.0
rlp==1.0.3
six==1.11.0
toolz==0.9.0
traitlets==4.3.2
urllib3==1.24.1
vyper==0.1.0b4
wcwidth==0.1.7
web3==4.8.1
websockets==6.0

What was wrong?

A little difficult to pin down. I am attempting to deploy a contract with the Infura Ropsten auto instance in a script. It looks like in web3/utils/transactions.py (line 77) that web3.eth.getBlock(web3.eth.blockNumber) is returning None, therefore block['gasLimit'] raises the exception. Trace:

---> 28         txn_hash = w3.eth.contract(**contract_interface).constructor(*args).transact(transact)
     29         print("Waiting for {} to mine...".format(txn_hash))
     30         contract_address = w3.eth.waitForTransactionReceipt(txn_hash).contractAddress

web3/utils/decorators.py in _wrapper(*args, **kwargs)
     12         def _wrapper(*args, **kwargs):
     13             if obj is not None:
---> 14                 return self.method(obj, *args, **kwargs)
     15             else:
     16                 return self.method(objtype, *args, **kwargs)

web3/contract.py in transact(self, transaction)
    854
    855         # TODO: handle asynchronous contract creation
--> 856         return self.web3.eth.sendTransaction(transact_transaction)
    857
    858     @combomethod

web3/eth.py in sendTransaction(self, transaction)
    261                 transaction,
    262                 'gas',
--> 263                 get_buffered_gas_estimate(self.web3, transaction),
    264             )
    265

web3/utils/transactions.py in get_buffered_gas_estimate(web3, transaction, gas_buffer)
     84     gas_estimate = web3.eth.estimateGas(gas_estimate_transaction)
     85
---> 86     gas_limit = get_block_gas_limit(web3)
     87
     88     if gas_estimate > gas_limit:

web3/utils/transactions.py in get_block_gas_limit(web3, block_identifier)
     76         block_identifier = web3.eth.blockNumber
     77     block = web3.eth.getBlock(block_identifier)
---> 78     return block['gasLimit']
     79
     80

TypeError: 'NoneType' object is not subscriptable

I've been getting really inconsistent interactions with this for the past little bit, so I am thinking it is Infura acting strangely, but some guards could be put in place to make sure this evaluates more as expected.

Not sure what (if anything) should be done about it.

[pipermerriam]

I'm pretty sure this is a common race condition bug that shows up when connected to infura. The call to web3.eth.blockNumber likely hit one of their nodes which had just received a new block, but the subsequent call to web3.eth.getBlock(...) hits another node which has not yet had that block propagated to it, and thus, it returns None. I'm thinking we should fix this at it's root by removing the functionality where None is returned when something isn't found in favor of raising an exception. This fixes the problem of mis-treating a return value that might be None by replacing it with something un-ignorable that blows up at the point in the code where the error was (basically, calls to web3.eth.getBlock would raise the exception, rather than the error only showing up later when block["gasLimit"] is accessed.

[stefanmendoza]

The call to web3.eth.blockNumber likely hit one of their nodes which had just received a new block, but the subsequent call to web3.eth.getBlock(...) hits another node which has not yet had that block propagated to it, and thus, it returns None.

I've been running into this lately now that I've started messing around with Infura. I think this makes a lot of sense. I always thought once you establish the Web3 instance, every call will use the same node.