Flask-socketio: start_background_task - Multiple Arguments (target, *args, **kwargs)

[hextrim]

Hi Miguel,

I tried to reach you on your blog site however its more reasonable to leave a note here. (please ignore my note on the blog)

I am using flask-socketio for a while now, but was using it in version Flask-SocketIO (0.3.7) - very old now.

I have a lot of specific tasks that run with "while" function for a reason.

Since I had some difficulties using the older version, I recently discovered an update the one of yours! :+1:

Within it is the so helpful start_background_task!

Previously I run my functions under @socketio.on() as follows:
@socketio.on('connect', namespace='/swxprogress'):
cct_swx('/dev/ttyUSB0', 'sw1usbsignal', 'sw1portsignal')
OR
@socketio.on('connect', namespace='/swxprobe')
def console_connect_test_sw1():
global thread_sw1
if thread_sw1 is None:
thread1 = Thread(target=cct_swx, args=('/dev/ttyUSB0', 'sw1usbsignal', 'sw1portsignal'),)

FUNCTION: def cct_swx(serial_port, sw_usbevent, sw_portevent):

For both using the very old gevent, flask, python-socketio, flask-socketio, gunicorn etc...
I was able to pass multiple arguments to the function from the call as detailed above.

Since I migrated to latest packages of the above (my app still works as a charm...) but the only way now to use my background functions which uses "while" is to use "start_background_task".

It seems that I can pass arguments to the function: start_background_task only in one way.
@socketio.on('connect', namespace='/swxprobe')
def console_connect_test_sw1():
global thread_sw1
with thread_lock:
if thread_sw1 is None:
thread_sw1 = socketio.start_background_task(target=cct_swx('/dev/ttyUSB1', 'sw2usbsignal', 'sw2portsignal'))

When I run it like that it seems to work but it produces some errors:
May 6 09:36:28 localhost gunicorn: Traceback (most recent call last):
May 6 09:36:28 localhost gunicorn: File "/var/www/iomate/iomateenv/lib/python2.7/site-packages/gevent/greenlet.py", line 538, in run
May 6 09:36:28 localhost gunicorn: self._report_error(sys.exc_info())
May 6 09:36:28 localhost gunicorn: File "/var/www/iomate/iomateenv/lib/python2.7/site-packages/gevent/greenlet.py", line 520, in _report_error
May 6 09:36:28 localhost gunicorn: self._exc_info = exc_info[0], exc_info[1], dump_traceback(exc_info[2])
May 6 09:36:28 localhost gunicorn: File "/var/www/iomate/iomateenv/lib/python2.7/site-packages/gevent/_tblib.py", line 370, in g
May 6 09:36:28 localhost gunicorn: return f(a)
May 6 09:36:28 localhost gunicorn: File "/var/www/iomate/iomateenv/lib/python2.7/site-packages/gevent/_tblib.py", line 425, in dump_traceback
May 6 09:36:28 localhost gunicorn: return dumps(tb)
May 6 09:36:28 localhost gunicorn: TypeError: 'NoneType' object is not callable
May 6 09:36:28 localhost gunicorn: Sun May 6 09:36:28 2018 failed with TypeError

In other hand when I try to run it like that:
@socketio.on('connect', namespace='/swxprobe')
def console_connect_test_sw1():
global thread_sw1
with thread_lock:
if thread_sw1 is None:
thread_sw1 = socketio.start_background_task(target=cct_swx, args=('/dev/ttyUSB1', 'sw2usbsignal', 'sw2portsignal'),)

It errors:
May 6 10:03:34 localhost gunicorn: Traceback (most recent call last):
May 6 10:03:34 localhost gunicorn: File "/var/www/iomate/iomateenv/lib/python2.7/site-packages/gevent/greenlet.py", line 536, in run
May 6 10:03:34 localhost gunicorn: result = self._run(self.args, *self.kwargs)
May 6 10:03:34 localhost gunicorn: TypeError: cct_swx() got an unexpected keyword argument 'args'
May 6 10:03:34 localhost gunicorn: Sun May 6 10:03:34 2018

The sample of my background task corresponding to the callable arguments:
def cct_swx(serial_port, sw_usbevent, sw_portevent):
console.port = serial_port
[..]
if console.is_open is True:
socketio.emit(sw_usbevent, {'data': 'CONNECTED'}, namespace='/swxprobe')
[..]
if console_input == '':
socketio.emit(sw_portevent, {'data': 'NOT READY '}, namespace='/swxprobe')

Is it something that I am not doing quite right here, is there a better way or an appropriate way to pass arguments, or the function cannot handle argument pass to socketio.emit within my background task.

[miguelgrinberg]

Here is how you need to call start_background_task:

thread_sw1 = socketio.start_background_task(cct_swx, '/dev/ttyUSB1', 'sw2usbsignal', 'sw2portsignal')

The first argument is the background function, the remaining ones are the arguments to pass to it.

[hextrim]

Hi Miguel,

Thank you for looking at it however it did not do anything other than:

@socketio.on('connect', namespace='/swxprobe')
def console_connect_sw():
global thread_sw1
with thread_lock:
if thread_sw1 is None:
thread_sw1 = socketio.start_background_task(target=cct_swx, '/dev/ttyUSB0', 'sw1usbsignal', 'sw1portsignal')
global thread_sw2
with thread_lock:
if thread_sw2 is None:
thread_sw2 = socketio.start_background_task(target=cct_swx, '/dev/ttyUSB1', 'sw2usbsignal', 'sw2portsignal')

May 7 10:12:52 ht-iomate-03 gunicorn: File "/var/www/iomate/iomateenv/lib/python2.7/site-packages/gevent/builtins.py", line 93, in __import__
May 7 10:12:52 ht-iomate-03 gunicorn: result = _import(args, *kwargs)
May 7 10:12:52 ht-iomate-03 gunicorn: File "/var/www/iomate/iomate/views.py", line 2137
May 7 10:12:52 ht-iomate-03 gunicorn: thread_sw1 = socketio.start_background_task(target=cct_swx, '/dev/ttyUSB0', 'sw1usbsignal', 'sw1portsignal')
May 7 10:12:52 ht-iomate-03 gunicorn: SyntaxError: non-keyword arg after keyword arg
May 7 10:12:52 ht-iomate-03 gunicorn: [2018-05-07 10:12:52 +0000] [24390] [ERROR] Exception in worker process
May 7 10:12:52 ht-iomate-03 gunicorn: Traceback (most recent call last):
May 7 10:12:52 ht-iomate-03 gunicorn: File "/var/www/iomate/iomateenv/lib/python2.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
May 7 10:12:52 ht-iomate-03 gunicorn: worker.init_process()
May 7 10:12:52 ht-iomate-03 gunicorn: File "/var/www/iomate/iomateenv/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 203, in init_process
May 7 10:12:52 ht-iomate-03 gunicorn: super(GeventWorker, self).init_process()
May 7 10:12:52 ht-iomate-03 gunicorn: File "/var/www/iomate/iomateenv/lib/python2.7/site-packages/gunicorn/workers/base.py", line 129, in init_process
May 7 10:12:52 ht-iomate-03 gunicorn: self.load_wsgi()
May 7 10:12:52 ht-iomate-03 gunicorn: File "/var/www/iomate/iomateenv/lib/python2.7/site-packages/gunicorn/workers/base.py", line 151, in load_wsgi
May 7 10:12:52 ht-iomate-03 gunicorn: self.reloader.add_extra_file(exc_val.filename)
May 7 10:12:52 ht-iomate-03 gunicorn: AttributeError: 'NoneType' object has no attribute 'add_extra_file'
May 7 10:12:52 ht-iomate-03 gunicorn: [2018-05-07 10:12:52 +0000] [24390] [INFO] Worker exiting (pid: 24390)
May 7 10:12:52 ht-iomate-03 gunicorn: [2018-05-07 10:12:52 +0000] [24385] [INFO] Shutting down: Master
May 7 10:12:52 ht-iomate-03 gunicorn: [2018-05-07 10:12:52 +0000] [24385] [INFO] Reason: Worker failed to boot.

[miguelgrinberg]

Remove the target= for the first argument.

[hextrim]

Hi Miguel,

That's now working well.

Sorry I overlooked it at first place.

Appreciate your help, and this is now resolved.

Thank you very much.

[markchou]

Here is how you need to call start_background_task:

thread_sw1 = socketio.start_background_task(cct_swx, '/dev/ttyUSB1', 'sw2usbsignal', 'sw2portsignal')

The first argument is the background function, the remaining ones are the arguments to pass to it.

working well,Thank you very much.