Flask-socketio: How can let "socketIO_client" send flask_socketio server "PING"?

[lgprobert]

Hi,

I experience same situation like: https://github.com/miguelgrinberg/Flask-SocketIO/issues/248, that I try to write a python client with socketIO_client to receive log messages from flask_socketio server where it retried from a container log (like tail -f) .

The flask_socketio server drops the client after 60 seconds (defined by ping_timeout). From the above issue link, I understand I should make socketIO_client sends flask socket server a ping message (probably after every log it received?), but I don't know how and can't find related documents from socketIO_client? Can anyone share some idea or very basic code here?

The other thing is because socket server retrieves a container log and emit everything it get to client in cyclic way. Even client aborts the socket connection by "ctrl-C" or pro programatically disconnect from socket server, I see socket still emit the logs it get. So how can I write the server side code to let flask_socketio only emit messages when the socket client connection is alive?

Thanks

[Hendrikto]

Hi @lgprobert,

As mentioned in the issue #248 you linked, you might want to turn on logging (add logger=True, engineio_logger=True to your flask_socketio.SocketIO() call) to see exactly what is going on.

The socketio server and client should handle keeping the connection alive themselves automatically. Normally you should not have to interfere:

The client should be sending a ping packet every 25 seconds, and this should be immediately responded with a pong packet back to the client.

Check the server logs and see if you have those ping/pong exchanges. If the server does not see a ping from the client in a 60 second interval, then it is going to throw away its session. That seems to be your problem, so I guess we need to figure out where are those pings.

If a client disconnects, you should get a disconnect event on the server ("Flask-SocketIO also dispatches connection and disconnection events." source). Upon disconnecting, a client should be automatically removed from all rooms it is in ("When a client disconnects it is removed from all the rooms it was in." source).

[lgprobert]

Hendrikto,

Thanks for the information, it helps a lot.

After a more tries, I understand more about socketio_client. When client does not emit an event, I can see ping/pong happens between server and client. Now the problem is on the emitting function @socketio.on('alog', namespace='/test'). This function does not end itself when a client is disconnected.

My code is like this:
Server side:

@socketio.on('alog', namespace='/test')
def test_connect():
    print("Welcome, alog received")
    logging_response = v1.read_namespaced_pod_log('zk1-zookeeper-0', 'default', pretty=False, follow=True, _preload_content=False, since_seconds = 10)
    for linedata in logging_response:
        linedata = linedata.decode('utf-8').rstrip()
        emit('alog_response', {'data': linedata})
        socketio.sleep(0)

Client side:

while True:
        print(datetime.datetime.now())
        test_namespace.emit('alog')
        socketIO.wait(seconds=1)

The above FOR LOOP is basically an infinite loop keep getting new logs from remote container and emit to client. Whenever the server side @socketio.on() decorator is triggered, it blocks any other things including the receiving of ping events that causes server does not know client is online. Another bad side effect is when server drops client connection given no ping messages are received, this function is still running on server side.

Here are the logs I got:

...
emitting event "alog_response" to a4abf41c28d34b1b977ecd1b93aee479 [/test]
a4abf41c28d34b1b977ecd1b93aee479: Sending packet MESSAGE data 2/test,["alog_response",{"data":"2018-08-06 07:11:16,098 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@827] - Processing ruok command from /127.0.0.1:56424"}]
emitting event "alog_response" to a4abf41c28d34b1b977ecd1b93aee479 [/test]
a4abf41c28d34b1b977ecd1b93aee479: Sending packet MESSAGE data 2/test,["alog_response",{"data":"2018-08-06 07:11:16,099 [myid:1] - INFO  [Thread-173118:NIOServerCnxn@1008] - Closed socket connection for client /127.0.0.1:56424 (no session established for client)"}]
emitting event "alog_response" to a4abf41c28d34b1b977ecd1b93aee479 [/test]
a4abf41c28d34b1b977ecd1b93aee479: Sending packet MESSAGE data 2/test,["alog_response",{"data":"2018-08-06 07:11:23,980 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /127.0.0.1:56478"}]
emitting event "alog_response" to a4abf41c28d34b1b977ecd1b93aee479 [/test]
a4abf41c28d34b1b977ecd1b93aee479: Client is gone, closing socket
emitting event "alog_response" to a4abf41c28d34b1b977ecd1b93aee479 [/test]
emitting event "alog_response" to a4abf41c28d34b1b977ecd1b93aee479 [/test]

Any way to fix this situation?

Thanks

[lgprobert]

It seems to me that comes down to two questions:

1. Can disconnection event (@socketio.on('disconnect', namespace='/test')) be used to stop long running emitting functions (@socketio.on('alog', namespace='/test'))?

2. How can I have a long running emitting function like the one in above code and have PING/PONG working on the mean time?

Thanks

[miguelgrinberg]

Event handlers aren't supposed to be long running. If you need an operation to run for a very long time, start a background task from the event handler. You can use socketio.start_background_task() for that.

[Hendrikto]

To elaborate on @miguelgrinberg's answer:

Socketio uses an asynchronous event based model. The event handler is supposed to react to events. With your infinite loop you keep it running indefinitely. Instead you should emit your log events from a background task started with socketio.start_background_task() as mentioned by @miguelgrinberg . To do this you will need to use socketio.emit() (which is different from the context-aware emit()) because there is no request context.

[lgprobert]

Hendrikto, miguelgrinberg

Thanks. This is exactly what I am looking for. One more question about socketio.start_background_task(). How is the life cycle of it? Is it the implementer responsible for terminating it when necessary? Again in my case, when clients disconnect, are there suitable methods, event handlers that I can leverage to stop the background tasks?

[lgprobert]

After changing my codes by following the great example app.py, it looks like all my troubles are gone. The background task seems destroyed when client is disconnected. This is great.

A little side effect when I tested with eventlet, which throws an error when server detect client disconnected. There is no error when I don't use eventlet. It seems that is problem of eventlet. The error is like this:

Exception ignored in: <Finalize object, dead>
Traceback (most recent call last):
  File "/usr/lib64/python3.4/multiprocessing/util.py", line 185, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/lib64/python3.4/multiprocessing/pool.py", line 535, in _terminate_pool
    cls._help_stuff_finish(inqueue, task_handler, len(pool))
  File "/usr/lib64/python3.4/multiprocessing/pool.py", line 772, in _help_stuff_finish
    inqueue.not_empty.acquire()
AttributeError: 'Queue' object has no attribute 'not_empty'

Thanks again for the great advice from @Hendrikto and @miguelgrinberg