Web3.py: websocket provider not working

[jfdelgad]

• Version: 4.5
• Python: 3.5
• OS: win

What was wrong?

impossible to connect to infura using websockets

I am using this to connect to infura:

from web3 import Web3
web3 = Web3(Web3.WebsocketProvider("wss://ropsten.infura.io/ws"))
web3.eth.blockNumber

No response, the console just become unresponsive. Tried two different computers in different networks, same result. Also used rinkeby and mainnet, same result.

[voith]

thanks for the report @jfdelgad.
firstly, the connection should timeout after sometime instead of just hanging and is noted in #956.
I tested my code on mainnet, rinkeby, ropsten and kovan. Below are the results.

on mainnet

In [31]: w3 = Web3(Web3.WebsocketProvider('wss://mainnet.infura.io/_ws'))

In [32]: w3.eth.blockNumber
Out[32]: 6184928

Notice the _ws in the uri.
However, when I remove the _ws and try wss://mainnet.infura.io/ws, it hangs.

In [33]: w3 = Web3(Web3.WebsocketProvider('wss://mainnet.infura.io/ws'))

In [34]: w3.eth.blockNumber
# hangs

then I tried wscat

➜  ~ wscat -c wss://mainnet.infura.io/ws

connected (press CTRL+C to quit)
> {"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id": 1}
< {"jsonrpc":"2.0","id":1,"result":"0x5e5ff9"}

Next I tried with the websockets library:

import asyncio
import json
import websockets

async def get_eth_blockNumber(uri):
    async with websockets.connect(uri) as websocket:
        request_data = {'jsonrpc': '2.0', 'method': 'eth_blockNumber', 'params': [], 'id': 1}
        await websocket.send(json.dumps(request_data))

        result = await websocket.recv()
        print(result)

In [9]: asyncio.get_event_loop().run_until_complete(get_eth_blockNumber('wss://mainnet.infura.io/_ws'))
{"jsonrpc":"2.0","id":1,"result":"0x5e6076"}

In [10]: asyncio.get_event_loop().run_until_complete(get_eth_blockNumber('wss://mainnet.infura.io/ws'))
{"jsonrpc":"2.0","id":1,"result":"0x5e6077"}


In [11]: asyncio.get_event_loop().run_until_complete(get_eth_blockNumber('wss://ropsten.infura.io/ws'))
{"jsonrpc":"2.0","id":1,"result":"0x3b2f70"}

This is a bug in web3. I don't know when I'll get time to get to this.

[ETH-Pantheon]

Thanks a lot, this is useful.

[FYJen]

Bump!

Same issue here except both ws and _ws hang for me. I am using web3 4.6 and python 3.6.5.

There is an official document from infura that the de facto WebSocket URL should be the one without underscore, ws. See here

I have tried on both mainnet and rinkey (w3.eth.blockNumber), and both networkers hang for ws and _ws.

[ankitchiplunkar]

Yes, I am suffering from the same issue.

[voith]

Will try my best to take a look at this tonight. But not promising anything.

[voith]

It turns out that this line is creating the problem:
https://github.com/ethereum/web3.py/blob/baaafca9bfbaa21844951310ff3be0109551c473/web3/providers/websocket.py#L99
encode_rpc_request(method, params) returns bytes which for some reason causes infura to barf.(Not sure yet if the problem is with infura, but the same code works fine with geth and parity)
https://github.com/ethereum/web3.py/commit/df465dc041149a5eecdaaefc71ccfeaf60ef56af seems to fix the issue, but I need to investigate further.

[voith]

So this indeed turns out to be a bug in infura.
To be sure about it I tried another websockets library, websocket-client. I have left the results of my tests over here.

Websocket supports input as both string and binary. However, the opcodes for both differ in the DataFrame. You can look at the implementation here or read it up in the docs.
However, when data is sent as bytes(opcode 2), it just hangs.

This is not a bug in web3 but a bug in infura.

We can still fix this in web3 as a temporary workaround, something like https://github.com/ethereum/web3.py/commit/df465dc041149a5eecdaaefc71ccfeaf60ef56af. But let @carver and @pipermerriam make this call.

[carver]

Sure, a decode() in the WebsocketProvider seems reasonable. I think Infura is the primary use case for WS at this point. But it definitely deserves an in-code comment explaining why, and maybe a link to the Infura issue saying that decode() could be removed if they decide to add support.

[voith]

Infura decided to fix this on their end: https://github.com/INFURA/infura/issues/134#issuecomment-417135301. I tested it myself, works fine!
@jfdelgad @ETH-Pantheon @ankitchiplunkar @torston @EvertKors @FYJen Can you guys retry again? The issue should be fixed.

@carver We can now close this issue.

[ankitchiplunkar]

@voith checked this on my end works fine.

[carver]

Nice @voith