Python-socketio: Simple question about rooms

[DevJhns]

Hi @miguelgrinberg !
Thx for your support, I got a lot of answers from your guides and comments.
Now there is stupidly simple question, maybe you can help?
My clients will make rooms/url links, how can I get it except passing from clientside on enter_room and storing in dicts/lists, are there any params in sessions, like sid?

[miguelgrinberg]

I think I'm going to need some additional clarification on your question, but I'm going to try to answer anyway.

The sid is always sent as a first argument to your event handlers, so you always know who originated an event you are handling. If you need to store information for your clients you have a few options. You can use dicts/lists if that works for your application, but I should note that if you ever need to scale your application to use multiple server instances, then holding data in memory becomes more complicated. You can also store data in a fast database such as Redis, or you can also use a database. The server provides a simple facility to create user sessions as well. These are indexed by the sid and are based on a dictionary of key/values.

[DevJhns]

I think I'm going to need some additional clarification on your question, but I'm going to try to answer anyway.

The sid is always sent as a first argument to your event handlers, so you always know who originated an event you are handling. If you need to store information for your clients you have a few options. You can use dicts/lists if that works for your application, but I should note that if you ever need to scale your application to use multiple server instances, then holding data in memory becomes more complicated. You can also store data in a fast database such as Redis, or you can also use a database. The server provides a simple facility to create user sessions as well. These are indexed by the sid and are based on a dictionary of key/values.

Thx a lot!
So sessions also stores in server memory... hmm
Yeah, I'm new at scaling apps, so your advice here would be awesome, at some point there is a possibility of highload I would be grateful if you point some way/s.
Idea is manager can create room and send link to clients or user can connect via widget from other site(but this is not topic now)
So room will be alive 1-2 hours max I think. 2-4 users per room, a lot of rooms at same time(lets imagine 100 - 10000) socketio, flask, gunicorn, eventlet + nginx. Previously I started new instance of app(Webrtc videochat mesh for now) with gunicorn for each room(:port)
Now I want as much rooms as possible per instance or all rooms in one instance if possible.
For testing this out I need to emit to rooms, so users, stats, features don't mess up.
Now I get room like this

@sio.on('create or join', namespace='/') def create_or_join(sid, data): sio.enter_room(sid, data)
and now I can store data in dicts, or can use session dicts, but seems in my situation this is not best solution?

[miguelgrinberg]

I think your solution is fine, but you should be aware that one day you may outgrow it. When you find that you need to scale your server to two instances storing in memory will have to be replaced with a shared storage, which could be Redis. As long as you are aware that there is a possible refactor down the road I think you will be fine. I suggest you create classes that provide access to your in-memory data structures, because then you can easily switch to a Redis solution without having to make changes all over your code.

[DevJhns]

I think your solution is fine, but you should be aware that one day you may outgrow it. When you find that you need to scale your server to two instances storing in memory will have to be replaced with a shared storage, which could be Redis. As long as you are aware that there is a possible refactor down the road I think you will be fine. I suggest you create classes that provide access to your in-memory data structures, because then you can easily switch to a Redis solution without having to make changes all over your code.

One last moment,
`sio = socketio.Server(async_mode='eventlet')
app = Flask(__name__)
app.config['SEND_FILE_MAX_AGE_DEFAULT'] = 0
app.wsgi_app = socketio.WSGIApp(sio, app.wsgi_app)

@sio.on('create or join', namespace='/')
def create_or_join(sid, data):
sio.enter_room(sid, data)
sio.save_session(sid, {'room': data})`

but when I emit
@sio.on('message', namespace='/') def messgage(sid, data): sio.emit('message', data=data, room=sio.get_session(sid)['room'])

I get KeyError: 'room' in different threads
should I share session dict to all threads, or I don't get idea of session in threads and need to use
class for data structures that will be global among all threads by default?
Can u link or give simple example of it?
Like
`#app

sio

app.wsgi= etc

Class Example(Something_with_wsgiapp_probably):
def Storing_rooms(self):
self.room_dict = {}`

Class should only be for storing/managing data or all code with sio.on, sio.emit should be in class?
Can I get this keyerror because of @sio.on operating too long?
Sometimes webrtc connection happens, sometimes I need to reload page this is strange for me, like it need some time or probably other request or right thread I guess(how to handle this then, store thread id somehow and wait only for it?) to write room to session thx again =)

[miguelgrinberg]

Are you sure you are storing the room key for all your clients? This is really simple, the dictionary that you write in the save_session call is retrieved when you use get_session.

[DevJhns]

Are you sure you are storing the room key for all your clients? This is really simple, the dictionary that you write in the save_session call is retrieved when you use get_session.

I do sio.save_session(sid, {'room': data}) on first emit from client to server no classes,
so it should be for all I guess.
Yet i get:

Exception in thread Thread-67:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(self.__args, *self.__kwargs)
File "/home/boss/app/appenv/local/lib/python2.7/site-packages/socketio/server.py", line 648, in _handle_event_internal
r = server._trigger_event(data[0], namespace, sid, data[1:])
File "/home/boss/app/appenv/local/lib/python2.7/site-packages/socketio/server.py", line 677, in _trigger_event
return self.handlers[namespace][event](args)
File "/home/boss/app/chat/webRTCserver.py", line 248, in color_border
sio.emit('color_border', {'id': sid, 'state': state}, room=sio.get_session(sid)['room'])
KeyError: 'room'

Exception in thread Thread-70:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(self.__args, *self.__kwargs)
File "/home/boss/app/appenv/local/lib/python2.7/site-packages/socketio/server.py", line 648, in _handle_event_internal
r = server._trigger_event(data[0], namespace, sid, data[1:])
File "/home/boss/app/appenv/local/lib/python2.7/site-packages/socketio/server.py", line 677, in _trigger_event
return self.handlers[namespace][event](args)
File "/home/boss/app/chat/webRTCserver.py", line 248, in color_border
sio.emit('color_border', {'id': sid, 'state': state}, room=sio.get_session(sid)['room'])
KeyError: 'room'

Exception in thread Thread-72:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(self.__args, *self.__kwargs)
File "/home/boss/app/appenv/local/lib/python2.7/site-packages/socketio/server.py", line 648, in _handle_event_internal
r = server._trigger_event(data[0], namespace, sid, data[1:])
File "/home/boss/app/appenv/local/lib/python2.7/site-packages/socketio/server.py", line 677, in _trigger_event
return self.handlers[namespace][event](args)
File "/home/boss/app/chat/webRTCserver.py", line 248, in color_border
sio.emit('color_border', {'id': sid, 'state': state}, room=sio.get_session(sid)['room'])
KeyError: 'room'

Exception in thread Thread-76:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(self.__args, *self.__kwargs)
File "/home/boss/app/appenv/local/lib/python2.7/site-packages/socketio/server.py", line 648, in _handle_event_internal
r = server._trigger_event(data[0], namespace, sid, data[1:])
File "/home/boss/app/appenv/local/lib/python2.7/site-packages/socketio/server.py", line 677, in _trigger_event
return self.handlers[namespace][event](args)
File "/home/boss/app/chat/webRTCserver.py", line 248, in color_border
sio.emit('color_border', {'id': sid, 'state': state}, room=sio.get_session(sid)['room'])
KeyError: 'room'

Exception in thread Thread-78:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 754, in run
self.__target(self.__args, *self.__kwargs)
File "/home/boss/app/appenv/local/lib/python2.7/site-packages/socketio/server.py", line 648, in _handle_event_internal
r = server._trigger_event(data[0], namespace, sid, data[1:])
File "/home/boss/app/appenv/local/lib/python2.7/site-packages/socketio/server.py", line 677, in _trigger_event
return self.handlers[namespace][event](args)
File "/home/boss/app/chat/webRTCserver.py", line 248, in color_border
sio.emit('color_border', {'id': sid, 'state': state}, room=sio.get_session(sid)['room'])
KeyError: 'room'

same for other emits from server to client, but when I reload page, all works

[miguelgrinberg]

Do you have your server on auto-reload when the code is changed? That would cause the sessions to all be dropped, since there is no permanent storage. I haven't seen your application, is it possible that it is designed in a way that the setting of the room in the session happens on a first connection, but not on a reconnection such as what occurs on the server restart?

[DevJhns]

Do you have your server on auto-reload when the code is changed? That would cause the sessions to all be dropped, since there is no permanent storage. I haven't seen your application, is it possible that it is designed in a way that the setting of the room in the session happens on a first connection, but not on a reconnection such as what occurs on the server restart?

Omg I'm so sorry @miguelgrinberg for wasting Your time!!! :(
It was my mistake, try except logic, on first call, server passes try and do except, so session doesn't save, but on second call dict with key exits and it do try block with session saving,
thank you for your help, and high quality product