Python-socketio: Hangs on connection with some socket-io clients

[bundabrg]

I'll reference this bug report for socketIO-client: Issue 118.

It appears that if a 'connect' event is captured on the server, and during its execution it emits another event, that some clients will struggle to connect. Tested with both the pythong socket-IO client and a dotnet client.

As 'connect' is a special event, returning False will cause a client to be disconnected. This means that any events that happen in it are added to the packet queue, and only once the event completes does the CONNECT packet get added. These other clients are only expecting the CONNECT and get confused with the event packets coming through.

Removing any emits from my connect handler fixes the python client. And applying a few small patches to handle Content-Type better (depending on b64 flag) fixes the dotnet client as well.

[miguelgrinberg]

Okay, I see the problem.

This is at the Socket.IO level only. For Engine.IO, I allow the connection before the connect handler is invoked (see here), but on this side the CONNECT packet goes after the connect handler is invoked.

I'll see if the order can be corrected, while still having a way to disconnect the user when the handler returns False.

[bundabrg]

If it helps, I fixed it by making the following changes:

• In engineio.socket.Socket._poll, after one packet is read I check if its an 'OPEN'. If so, I return that packet, else proceed to see if any other packets are available (which normal packs them together, but breaks on the OPEN). This delays the CONNECT correctly.
• In engineio.socket._websocket_handler I commented out 'self.queue.join() # flush the queue first', as there is no consumer this gets stuck.
• Lastly (but not related to this bug) I edited engineio.Server._ok so it checks the b64 flag and returns a content type of tex/plain if its set, else application/octet-stream. This fixes the dotnet socket.io client connecting.

And as I type this all out I realise they're all in engineio. Well hope it gives you some hints where you could look anyway.

[catmeme]

Thanks for looking into this, I believe I have a related issue with artillery. All my tests fail to connect properly, and what you have described reflects my symptoms. I found this by searching the issues for "b64", which was the only notable difference in the GET request between artillery, and the socket-io Flask demo client.

[miguelgrinberg]

@catmeme Can you tell me more about the failure you are seeing? Is it that the polling response when using base64 encoding comes with a content type of application/octet-stream instead of text/plain?

Or actually, I just fixed the b64 content type. Please try upgrading package python-engineio to version 1.1.1 and let me know if there are any other problems.

[catmeme]

I will give it a shot.

Just to follow up:

The server receives the following request from the Flask socket.io example

GET /socket.io/?EIO=3&transport=polling&t=1485126264966-0

This works as expected.

However, Artillery starts with the following request will results in a failure (which I'm still gathering information on, very new to socket.io)

GET /socket.io/?EIO=3&transport=polling&t=Ld8QsPf&b64=1

[catmeme]

Thanks, I verified upgrading to engineio 1.1.1 has resolved the error I was seeing with Artillery.

172.28.0.1 - - [23/Jan/2017:18:12:10] "GET /socket.io/?EIO=3&transport=polling&t=LdCXpX1&b64=1 HTTP/1.1" 200 - "-" "node-XMLHttpRequest"
172.28.0.1 - - [23/Jan/2017:18:12:10] "POST /socket.io/?EIO=3&transport=polling&t=LdCXpXJ&b64=1&sid=ce45d5b514f74326afdf71c383d9e368 HTTP/1.1" 200 - "-" "node-XMLHttpRequest"
172.28.0.1 - - [23/Jan/2017:18:12:10] "GET /socket.io/?EIO=3&transport=polling&t=LdCXpXK&b64=1&sid=ce45d5b514f74326afdf71c383d9e368 HTTP/1.1" 200 - "-" "node-XMLHttpRequest"
2017-01-23 18:12:10 [26] [ERROR] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/site-packages/gunicorn/workers/async.py", line 45, in handle
    self.handle_request(listener, req, client, addr)
  File "/usr/local/lib/python3.5/site-packages/gunicorn/workers/async.py", line 102, in handle_request
    resp.close()
  File "/usr/local/lib/python3.5/site-packages/gunicorn/http/wsgi.py", line 369, in close
    self.send_headers()
  File "/usr/local/lib/python3.5/site-packages/gunicorn/http/wsgi.py", line 284, in send_headers
    tosend = self.default_headers()
  File "/usr/local/lib/python3.5/site-packages/gunicorn/http/wsgi.py", line 265, in default_headers
    elif self.should_close():
  File "/usr/local/lib/python3.5/site-packages/gunicorn/http/wsgi.py", line 198, in should_close
    if self.status_code < 200 or self.status_code in (204, 304):
AttributeError: 'Response' object has no attribute 'status_code'
172.28.0.1 - - [23/Jan/2017:18:12:10] "GET /socket.io/?EIO=3&transport=websocket&sid=ce45d5b514f74326afdf71c383d9e368 HTTP/1.1" 500 - "-" "-"

However, I now see an error in the output, but I think this error is related to #38

[miguelgrinberg]

@catmeme great. The status_code error is a known issue with gunicorn. It has been fixed already, but they haven't released a new version since the fix. You should be fine if you install their master branch.

[catmeme]

@miguelgrinberg thanks for the tip, I'll look into that.

[moizqureshi]

@bundabrg Thanks! I was having same issue with my client connects being slow and hanging and never thought to realize that having emits on the server-side connect events was causing this. Removing such emits there fixed my issue.

[adiun]

I was having the same issue with py-sio 1.8.0 and py-eio 2.0.1, but didn't have any emits on server-side connect events. The workaround from @bundabrg fixed my connection issue too - Thanks!

[sphh]

Is this problem resolved yet? I have the same problem. Like @catmeme I use the Flask socket.io example. The server output is as follows:

127.0.0.1 - - [04/Mar/2018 21:02:37] "GET /socket.io/?EIO=3&transport=polling&t=1520197357334-0 HTTP/1.1" 200 381 0.022752
(2270) accepted ('127.0.0.1', 55518)

The client does not see the "connect" message and cannot act accordingly. If I disconnect the client, it sees the "disconnect" message (so I assume the connection between the server and client works).

My environment:
Python 3.4
python-socketio installed via pip, so it should have the latest versions:
engineio 2.0.2
socketio 1.8.4

As a client I use the socketIO-client (https://github.com/invisibleroads/socketIO-client), also freshly installed via pip.

I tried to apply the workaround from @bundabrg, but don't know what to apply as a first step. @bundabrg could you please explain what you changed? Thanks.

[adiun]

@sphh - I interpreted that first step from @bundabrg 's comment like this, by monkey-patching engineio.asyncio_socket.AsyncSocket.poll by adding another branch to the block that starts with "Wait for packets to send to the client"

# NOTE: (Patch) return the first packet if it's OPEN. Then return the others
elif packets[0].packet_type == 0:
    return [packets[0]]

I also monkey patched asyncio_socket.AsyncSocket._websocket_handler re. the second step. It's been working for me for the last few months.

[sphh]

Thank you @adiun for your help! That works. I applied these changes in engineio.socket.Socket.poll and engineio.socket.Socket._websocket_handler and it seems to work.

Maybe @miguelgrinberg can add these changes to the next release (if there a no negative side effects)?

[miguelgrinberg]

I have finally found what I think is a good solution to this problem. Starting from release 4.0 the connection is established before the connect event handler is invoked. If the handler returns False, then a disconnect is issued after it returns.

If you need this functionality, you need to initialize the server with the always_connect=True option. The default for this option is False, which keeps the current way of working, which is appropriate when the connect handler does not emit events back to the client.