Web3.py: Difficulty to set up infura RPCprovider

[jdetychey]

• Version: x.x.x
• Python: 2.7/3.4/3.5
• OS: linux (cloud9)

What was wrong?

I registred on Infura, got my token GNAGNAGNA
I read the doc and find infura mention at:
http://web3py.readthedocs.io/en/latest/managers.html

web3 = Web3(Web3.RPCProvider(host='mainnet.infura.io', path='your-infura-access-key'))

I have been trying to use syntaxes:

web3 = Web3(Web3.RPCProvider(host='mainnet.infura.io', path='GNAGNAGNA'))
web3 = Web3(Web3.RPCProvider(host='ropsten.infura.io', path='GNAGNAGNA'))
web3 = Web3(Web3.RPCProvider(host='https://mainnet.infura.io', path='GNAGNAGNA'))
web3 = Web3(Web3.RPCProvider(host='http://mainnet.infura.io', path='GNAGNAGNA'))

none of them seems to work although on nodejs:

web3 = new Web3(new Web3.providers.HttpProvider("https://ropsten.infura.io/GNAGNAGNA"));

works properly

Cute Animal Picture

happy holidays !

[jdetychey]

Full code:

from web3 import Web3, RPCProvider
web3 = Web3(Web3.RPCProvider(host='ropsten.infura.io', path='GNAGNAGNA',port=443,ssl=True))
value = web3.eth.gasPrice
print value

Returns the error:

Traceback (most recent call last):
File "/home/ubuntu/workspace/test/web3call.py", line 4, in
value = web3.eth.gasPrice
File "/usr/local/lib/python2.7/dist-packages/web3/utils/functional.py", line 22, in inner
value = fn(args, *kwargs)
File "/usr/local/lib/python2.7/dist-packages/web3/eth.py", line 91, in gasPrice
return self.web3._requestManager.request_blocking("eth_gasPrice", [])
File "/usr/local/lib/python2.7/dist-packages/web3/providers/manager.py", line 37, in request_blocking
response_raw = self.provider.make_request(method, params)
File "/usr/local/lib/python2.7/dist-packages/web3/providers/rpc.py", line 54, in make_request
response = client.post(self.path, body=request_data)
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 218, in post
return self.request(METHOD_POST, request_uri, body=body, headers=headers)
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 175, in request
sock = self._connection_pool.get_socket()
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 136, in get_socket
return self._create_socket()
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 107, in _create_socket
self.after_connect(sock)
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 200, in after_connect
match_hostname(sock.getpeercert(), self._host)
File "/usr/local/lib/python2.7/dist-packages/backports/ssl_match_hostname/__init__.py", line 151, in match_hostname
% (hostname, dnsnames[0]))
backports.ssl_match_hostname.CertificateError: hostname 'ropsten.infura.io' doesn't match 'api-consensysnet.infura.io'

[jdetychey]

upgraded web3 via pip.
Tested different way to use "web3 = ":

from web3 import Web3, RPCProvider
web3 = Web3(RPCProvider(host='https://ropsten.infura.io/GNAGNAGNA',port='8545'))

web3 = Web3(Web3.RPCProvider(host='ropsten.infura.io', path='GNAGNAGNA',port=443,ssl=True))

web3 = Web3(Web3.RPCProvider(host='ropsten.infura.io', path='GNAGNAGNA'))

value = web3.eth.gasPrice
print value

All of them exit with code 1. Here is the full log:

Traceback (most recent call last):
File "/home/ubuntu/workspace/test/web3call.py", line 1, in
from web3 import Web3, RPCProvider
File "/usr/local/lib/python2.7/dist-packages/web3/__init__.py", line 5, in
from web3.main import Web3
File "/usr/local/lib/python2.7/dist-packages/web3/main.py", line 19, in
from web3.providers.tester import (
File "/usr/local/lib/python2.7/dist-packages/web3/providers/tester.py", line 5, in
from testrpc.rpc import RPCMethods
ImportError: No module named testrpc.rpc

Edit: cute animal picture:

[pipermerriam]

@jdetychey try upgrading to 3.4.2. 3.4.1 had a dependency bug that was just fixed.

[jdetychey]

upgrading solved the testrpc issue but the same code as above now has a different error:

Traceback (most recent call last):
File "/home/ubuntu/workspace/test/web3call.py", line 8, in
value = web3.eth.gasPrice
File "/usr/local/lib/python2.7/dist-packages/web3/utils/functional.py", line 22, in inner>
value = fn(args, *kwargs)
File "/usr/local/lib/python2.7/dist-packages/web3/eth.py", line 97, in gasPrice
return self.web3._requestManager.request_blocking("eth_gasPrice", [])
File "/usr/local/lib/python2.7/dist-packages/web3/providers/manager.py", line 41, in request_blocking
response_raw = self.provider.make_request(method, params)
File "/usr/local/lib/python2.7/dist-packages/web3/providers/rpc.py", line 71, in make_request
response = client.post(self.path, body=request_data)
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 218, in post
return self.request(METHOD_POST, request_uri, body=body, headers=headers)
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 175, in request
sock = self._connection_pool.get_socket()
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 136, in >get_socket
return self._create_socket()
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 94, in >_create_socket
sock_infos = self._resolve()
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 68, in _resolve
family, 0, gevent.socket.SOL_TCP)
File "/usr/local/lib/python2.7/dist-packages/gevent/_socketcommon.py", line 268, in getaddrinfo
return get_hub().resolver.getaddrinfo(host, port, family, socktype, proto, flags)
File "/usr/local/lib/python2.7/dist-packages/gevent/resolver_thread.py", line 64, in getaddrinfo
return self.pool.apply(_socket.getaddrinfo, args, kwargs)
File "/usr/local/lib/python2.7/dist-packages/gevent/pool.py", line 300, in apply
return self.spawn(func, args, *kwds).get()
File "/usr/local/lib/python2.7/dist-packages/gevent/event.py", line 373, in get
return self.get(block=False)
File "/usr/local/lib/python2.7/dist-packages/gevent/event.py", line 363, in get
return self._raise_exception()
File "/usr/local/lib/python2.7/dist-packages/gevent/event.py", line 343, in _raise_exception
reraise(self.exc_info)
File "/usr/local/lib/python2.7/dist-packages/gevent/threadpool.py", line 207, in _worker
value = func(args, **kwargs)
socket.gaierror: [Errno -2] Name or service not known
Process exited with code: 1

[pipermerriam]

So the issue you are seeing is likely because of a trailing / on you host. If you remove that you can get back to the previous SSL error. The issue is from geventhttpclient which I'm less than satisfied with right now as they aren't doing SSL correctly. This is on my radar but it may be next week before I can really get to it.

[pipermerriam]

Ok, I have a workaround for you.

from web3 import Web3, KeepAliveRPCProvider
from web3.providers.rpc import _client_cache
from geventhttpclient.connectionpool import ConnectionPool
from gevent import ssl

def monkeypatched_create_tcp_socket(*args, **kwargs):
     sock = ConnectionPool._create_tcp_socket(None, *args, **kwargs)
     ctx = ssl.create_default_context()
     wrapped_sock = ctx.wrap_socket(sock, server_hostname='ropsten.infura.io')
     return wrapped_sock

web3 = Web3(KeepAliveRPCProvider(host='ropsten.infura.io', port=443, ssl=True))
cached_client = _client_cache['ropsten.infura.io:443']
cached_client._connection_pool._create_tcp_socket = monkeypatched_create_tcp_socket

web3.eth.blockNumber

It's gross but it works.

[jdetychey]

well unfortunately that's it yet, maybe that's because I am using cloud9.

Important: use os.getenv(PORT, 8080) as the port and os.getenv(IP, 0.0.0.0) as the host in your scripts!

Traceback (most recent call last):
File "/home/ubuntu/workspace/test/web3call.py", line 33, in
value = web3.eth.blockNumber
File "/usr/local/lib/python2.7/dist-packages/web3/utils/functional.py", line 22, in inner
value = fn(args, *kwargs)
File "/usr/local/lib/python2.7/dist-packages/web3/eth.py", line 113, in blockNumber
return self.web3._requestManager.request_blocking("eth_blockNumber", [])
File "/usr/local/lib/python2.7/dist-packages/web3/providers/manager.py", line 41, in request_blocking
response_raw = self.provider.make_request(method, params)
File "/usr/local/lib/python2.7/dist-packages/web3/providers/rpc.py", line 160, in make_request
response = self.client.post(self.path, body=request_data)
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 218, in post
return self.request(METHOD_POST, request_uri, body=body, headers=headers)
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 175, in request
sock = self._connection_pool.get_socket()
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 136, in >get_socket
return self._create_socket()
File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 119, in >_create_socket
raise first_error
TypeError: unbound method _create_tcp_socket() must be called with ConnectionPool instance as >first argument (got NoneType instance instead)

I ll try from another python VM tomorrow

[pipermerriam]

Hrm, can you try this updated version

from web3 import Web3, KeepAliveRPCProvider
from web3.providers.rpc import _client_cache
from gevent import ssl


web3 = Web3(KeepAliveRPCProvider(host='ropsten.infura.io', port=443, ssl=True))
cached_client = _client_cache['ropsten.infura.io:443']
original_create_tcp_socket = cached_client._connection_pool._create_tcp_socket

def monkeypatched_create_tcp_socket(*args, **kwargs):
     sock = original_create_tcp_socket(*args, **kwargs)
     ctx = ssl.create_default_context()
     wrapped_sock = ctx.wrap_socket(sock, server_hostname='ropsten.infura.io')
     return wrapped_sock

cached_client._connection_pool._create_tcp_socket = monkeypatched_create_tcp_socket

web3.eth.blockNumber

[jdetychey]

Not working yet unfortunately:

Traceback (most recent call last):
  File "/home/ubuntu/workspace/test/web3call.py", line 35, in <module>
    web3.eth.blockNumber
  File "/usr/local/lib/python2.7/dist-packages/web3/utils/functional.py", line 22, in inner
    value = fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/web3/eth.py", line 113, in blockNumber
    return self.web3._requestManager.request_blocking("eth_blockNumber", [])
  File "/usr/local/lib/python2.7/dist-packages/web3/providers/manager.py", line 41, in request_blocking
    response_raw = self.provider.make_request(method, params)
  File "/usr/local/lib/python2.7/dist-packages/web3/providers/rpc.py", line 160, in make_request
    response = self.client.post(self.path, body=request_data)
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 218, in post
    return self.request(METHOD_POST, request_uri, body=body, headers=headers)
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/client.py", line 175, in request
    sock = self._connection_pool.get_socket()
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 136, in get_socket
    return self._create_socket()
  File "/usr/local/lib/python2.7/dist-packages/geventhttpclient/connectionpool.py", line 119, in _create_socket
    raise first_error
AttributeError: 'module' object has no attribute 'create_default_context'

[jdetychey]

BTW: the library works like charm when using:
web3 = Web3(KeepAliveRPCProvider(host='localhost', port='8545'))

^/^

[pipermerriam]

@jdetychey I just release 3.5.0 which does a few things, one of which is removal of gevent as a hard dependency. The other is deprecating RPCProvider and KeepAliveRPCProvider. Both are now replaced by the much simpler to use web3.providers.rpc.HTTPProvider.

So, there is some nuance to all of this because the library will now use standard library by default where previously it would use gevent. An added benefit of this is that if you are not using gevent (which is now the default) the library will now use requests to make the RPC calls instead of the geventhttpclient which is frustratingly broken when it comes to certain SSL configurations.

So, try the following.

from web3 import Web3
from web3.providers.rpc import HTTPProvider

web3 = Web3(HTTPProvider('https://mainnet.infura.io'))

If you want to use infura with your API key then you can just add that path to the provided URI string that you pass into HTTPProvider.

[edmundedgar]

I had the same problem and I can confirm that the HTTPProvider method is working for me, at least as far as retrieving web3.eth.blockNumber. I tried with and without my API token on the end of the URL, and they both worked.

It then proceeds to crap out with requests.exceptions.HTTPError: 405 Client Error: Method Not Allowed for url: https://ropsten.infura.io/ when I get to the next step and try to call pastEvents() on it, but I think that's web3.py behaving correctly, and Infura not wanting to serve me pastEvents.

[pipermerriam]

It then proceeds to crap out with requests.exceptions.HTTPError: 405 Client Error: Method Not Allowed for url: https://ropsten.infura.io/ when I get to the next step and try to call pastEvents() on it, but I think that's web3.py behaving correctly, and Infura not wanting to serve me pastEvents.

This sounds like expected behavior but I would confirm with the Infura team. I do know that currently infura doesn't support filters but at some point in the not to far past you could still hit those endpoints.