Python-socketio: Spawned emitting - class based namespace - thread still alive

[MFlyer]

Hi @miguelgrinberg ,
maybe it's related to gevent.spawn but having a strange behaviour while emitting from a class namespace (working on gevent supervisored):

on user connection we set a connected = True, send a welcome message (using room = sid too) then we gevent.spawn a func; this func emits to connected client (always using its sid) and has a gevent.sleep(x) to autoemit under a while connected statement. On disconnect we set our connected = False, client get disconnected, and here comes the fun:
if user move to another backbone view then back to that page all is ok (client connects, client disconnects, new client connection)
if user refresh (F5) our page client disconnects, new client connection, but process is alive and emitting.

Maybe related to handling page refresh? probably, but this doesn't solve the infinite emits against a disconnected client

Mirko

[miguelgrinberg]

Are your clients websocket or long-polling? When a websocket client refreshes the page, there is an immediate disconnection notification on the server side, but long-polling does not work that way because there isn't a permanent connection. For long-polling, the server declares a client disconnected after a minute passes without receiving any requests.

[MFlyer]

Ref. https://github.com/MFlyer/rockstor-core/blob/78d80c8b8688945aeb75cbaf870119224de2b380/src/rockstor/smart_manager/data_collector.py#L625

On line 769 i register that namespace.
Clients nicely upgrade to websocket and on disconnect actually it seems they get disconnected (had some debug and on disconnection the on_disconnect event is fired), but the spawned func (line 635) still emits to dead client (check this -> https://github.com/rockstor/rockstor-core/issues/1522#issuecomment-260363741): adding room = sid and disconnecting via sid don't solve, so maybe related to gevent not killing previous job (pls note migrating from gevent.socketio to python-socketio so probably gevent.socketio add a kind of "autokill" on pending threads)

Other note: if not emitting with room = sid, sid "B" receives sid "A" data too; I don't think it's client related (all connection get a force new connection : true), but on backend gevent.spawn staying alive

M.

[miguelgrinberg]

Oh, I see. Thanks for pointing me to the code. The misunderstanding on your part is that there isn't a Namespace instance per client. The namespace object is a singleton, so you cannot keep client state in it, like your start attribute. What you need to do is build your own per-client storage, which could be a global dictionary, for example, indexed by the client session id.

[MFlyer]

Thanks @miguelgrinberg , I had this suspicion and nice to read you confirm it, now I know how to solve it!
My suggestion: add this note somewhere on docs, specially for those coming from gevent.socketio (it seems a dead project but I'm quite sure it had a kind of client state handling and being gevent specific it had a good integration granting spawned funcs autokilled!)

Thanks again
Mirko

[MFlyer]

Hi @miguelgrinberg ,
I know this was closed, but find it stupid to have a clone of it.

Finally on @Rockstor had a "zombies killer" with this subclass extending socketio.Namespace and every namespace with something like this:

class ServicesNamespace(RockstorIO):

    start = False

    def on_connect(self, sid, environ):

        self.emit('connected', {
            'key': 'services:connected', 'data': 'connected'
        })
        self.start = True
        self.spawn(self.send_service_statuses, sid) # <- use newly created spawn

    def on_disconnect(self, sid):

        self.cleanup(sid) # <- grant killing all "zombies"
        self.start = False

Probably it would be nice to have same code extending both gevent & eventlet on class namespaces and I'd like to help and code it to, do you have any suggetion?
Mirko

[MFlyer]

Hi @miguelgrinberg ,
did you have time to check this?
BR
Mirko

[miguelgrinberg]

@MFlyer not sure how to code this in a generic way, because how an application manage its threads is really not standardized, each project does it in its own way. Killing threads by force, in principle, is not a good strategy, normally you signal the thread that it needs to exit.

[MFlyer]

Hi @miguelgrinberg , going to explain my idea:

creating a new python-socketio server users usually have
sio = socketio.Server(async_mode='myasync_mode') with myasync_mode = eventlet / gevent / gevent_uwsgi or same order loading if myasync_mode omitted

checking current async_mode value we can guess users having eventlet async using eventlet threads, gevent => gevent threads, etc, so we can al least grant 2 default methods (cleanup & spawn) like on Rockstor code

class RockstorIO(socketio.Namespace):
    "RockstorIO socketio.NameSpace SubClass"

    def __init__(self, *args, **kwargs):

        super(RockstorIO, self).__init__(*args, **kwargs)
        self.threads = {}

    def cleanup(self, sid):

        if sid in self.threads:
            gevent.killall(self.threads[sid])
            del self.threads[sid]

    def spawn(self, func, sid, *args, **kwargs):

        thread = gevent.spawn(func, *args, **kwargs)
        if sid in self.threads:
            self.threads[sid].append(thread)
        else:
            self.threads[sid] = [thread]

Every user can use these 2 (example self.spawn instead of gevent.spawn), but not forced to have them (example overwrite default threads handlers spawn & cleanup)

[miguelgrinberg]

@MFlyer to start a background thread you can use socketio.start_background_task() function. That works as you describe, the function checks the async mode in use and then calls the appropriate function to spawn the thread or greenlet. There is also socketio.sleep() which works in the same way.

Killing threads though, is a bad practice. It isn't supported on some platforms, and in general is not a good idea because it may leak resources. So I think this should be left to the application, there is no standard way to stop a thread other than signaling the thread to end, and this signaling can be done in a number of different ways that are application specific.