Flask-socketio: How to dynamically get current user list by using session?

[shisaq]

Hi Miguel, I'm a junior developer, I'm sorry to make you spend some time on this question...

• I'm using Flask-SocketIO to try to build a chat server _without_ using any database or user list stored in the back-end.
• I just simply store the username into session and authenticate the username info by Flask-Login, so that I can use SocketIO to get the current_user.

So here is the question:

How could I get the current user list by dynamically retrieving the sessions from different users, in order to let every user who has logged in see the list?

Thank you for reading this, I would appreciate it if you could give me some handy tips 😃😃😃

[miguelgrinberg]

Flask-SocketIO does not provide the list of connected users, because there are situations in which getting the list is not a trivial operation. On the other side, even though yours does not, most applications have some sort of globally accessible storage such as a database, which makes keeping a user list trivial at the application level.

If you don't want to maintain a list within your application then I'm afraid there is no good way to get a list. You can always look inside the internal structures held by the Socket.IO server to find the users, but this is not something I ever intend to support, so it will be a solution that could easily break in the future.

[shisaq]

@miguelgrinberg I got it. Thank you for your rapid response!

Since the internal structures of the Socket.IO server may be quite difficult for me, I think the best way would still be storing user data into a list or making a room to handle usernames.

[miguelgrinberg]

Actually I do have another (crazy) idea.

You could have each user broadcast the nickname to everybody else. This can be done by starting a little background thread each time a user connects. Inside the thread you can emit the nickname periodically.

The clients are then in charge if maintaining a user list, built from all these name broadcasts. They will need to check how long ago a nickname was received, and remove any names that aren't constantly broadcasting.

So there you go, no storage at all on the server, just a bunch of threads!

[shisaq]

Wow, it looks like a hack way!!

But if I emit the nickname periodically, the list will not be continually updated, which may occur some abnormal bugs, like A finds B in the list, but A fails to chat with B because actually B has already logged out...

[miguelgrinberg]

The idea of sending names repeatedly is to give new client the chance to build their own user list.

When a user goes away, you can have it broadcast a final message to all clients, and then clients can remove that user immediately. I think it still is a good idea to check and remove users that fail to broadcast in a while, though.

[shisaq]

@miguelgrinberg I'll try this solution! How often do you think it's perfect to send names, by considering both the latency and the server load?

[miguelgrinberg]

@shisaq it really depends on how many clients you can have. If you just have a handful of them, then sending names ever 5 seconds should be fine, and that ensures that a new client that connects will have the complete user list in at most 5 seconds. If you have lots of clients, then the traffic generated by these broadcasts might be significant, so you may want to make them less frequent.

[shisaq]

Hi @miguelgrinberg , actually I'm trying to use a background thread, in order to send the nickname periodically, but I met a problem:

How could I get access to visit the session when I'm currently in the background thread?

Here is my related code:

thread = None

def background_thread():
    count = 0
    while True:
        print 'I am in!!!'
        socketio.sleep(5)
        count += 1
        socketio.emit('update_list',
                      {'data': session['user'], 'count': count}, broadcast=True)

@socketio.on('connect')
def user_connect():
    global thread
    if thread is None:
        thread = socketio.start_background_task(target=background_thread)
    emit('update_list', {'data': session['user'], 'count': 0}, broadcast=True)

I got an error Working outside of request context.

I believe it's because every session must face one client, but the background thread actually has no specific client with it.

Is there any way I can do to broadcast the nicknames from the sessions? Thanks!

[miguelgrinberg]

The socketio.emit() function does not require a context, so I'm not sure what in the thread's code is complaining. Do you have a stack trace for the error?

[shisaq]

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/shisaq/Coding/pythonlearn/flask/chat-server/run.py", line 34, in background_thread
    {'data': session['user'], 'count': count}, broadcast=True)
  File "/Users/shisaq/Coding/pythonlearn/flask/chat-server/venv/lib/python2.7/site-packages/werkzeug/local.py", line 373, in <lambda>
    __getitem__ = lambda x, i: x._get_current_object()[i]
  File "/Users/shisaq/Coding/pythonlearn/flask/chat-server/venv/lib/python2.7/site-packages/werkzeug/local.py", line 302, in _get_current_object
    return self.__local()
  File "/Users/shisaq/Coding/pythonlearn/flask/chat-server/venv/lib/python2.7/site-packages/flask/globals.py", line 37, in _lookup_req_object
    raise RuntimeError(_request_ctx_err_msg)
RuntimeError: Working outside of request context.

This typically means that you attempted to use functionality that needed
an active HTTP request.  Consult the documentation on testing for
information about how to avoid this problem.

[miguelgrinberg]

Ah, yes, I missed the session reference, that requires a context, since it is obviously attached to a given client. What's the reason you have a username in a message that is broadcasted to all your clients?

[shisaq]

Actually, it's because I want to make every online client have the current client list. Especially as you said before, "_The idea of sending names repeatedly is to give a new client the chance to build their own user list._"

But I have a requirement limit below:

_User data must not persist on the server. The server should not have a database, and the server should not store any user data in memory (i.e. Don’t do something like this users= []). However, you are free to use SocketIO rooms._

According to this requirement, could you come up with any good idea? Thanks 😄

[miguelgrinberg]

Yeah, I remember what I told you a few days ago, but the way you coded the thread does not map to that. You have only one thread for your entire application, not one thread for each active user.

If you make a thread per user, then you can pass the user name as an argument to the thread, so that each thread has the user to broadcast in its own environment, without having to read it from the user session.

[shisaq]

I pretty much want to do as you say, but I'm not sure how to make a thread per user... Could you give me a simple code sample or related docs? I'm sorry but this is really hard for me... Thanks!

[miguelgrinberg]

Example:

@socketio.on('connect')
def user_connect():
    session['thread'] = socketio.start_background_task(background_thread, session['user'])

This sends the user as an argument to your thread function (you'll have to add the argument on your thread function, obviously). It also stores the thread in the user session, so that you can then access it when you need to stop it.

[shisaq]

It works! Thank you for your help! I feel extremely excited now! It's really a nice idea to store the thread into the session, I never thought about it before! @miguelgrinberg

[geasyheart]

I'm sorry to bother you again,As you said:

@socketio.on('connect')
def user_connect():
    session['thread'] = socketio.start_background_task(background_thread, session['user'])
I don't know what this means. Here's my code:

class Joined(Namespace):
    @staticmethod
    def background_thread(user):
        user_list = set()
        while 1:
            socketio.sleep(3)
            user_list.add(user)
            count = len(user_list)
            socketio.emit('message', {"message": "user_list:{},numbers:{}".format(user_list, count)}, broadcast=True,
                          namespace='/chat')

    def on_connect(self):
        print(request.sid)
        session['thread'] = socketio.start_background_task(self.background_thread, session.get('name'))

I think it is in each user session stored in a thread,so i created two users, the names are user1,user2, but the result is:

...
user_list:{'user1'},numbers,1
user_list:{'user2'},numbers,1
user_list:{'user1'},numbers,1
user_list:{'user1'},numbers,1
user_list:{'user2'},numbers,1
...

how should i make it become

 ...
 user_list:{'user1','user2'},numbers,2
 ...

I'm sorry but this is really hard for me,thank you!

[miguelgrinberg]

The code that you are using starts a thread per client. You want a single thread, so putting the start_background_thread call in the connect handler is not the correct approach. You can start the thread right before you call socketio.run(), for example. The user set will have to be a global variable, and in the connect handler you just add the new user to that set.