Flask-socketio: works well in uwsgi, but error in uwsgi-gevent

[Pang654]

Hi Miguel,
I am using flask-flask_socketio-uwsgi-gevent. I am getting module not callable when gevent is allowing in config. I have import gevent and monkey patch it.

Here is my configuration:

module = wsgi:application


processes = 4
threads = 2
plugin = python38

pidfile = uwsgi.pid


gevent = 10
master = true


http = 0.0.0.0:5000
http-websockets = true
socket = PloutosBackend.sock

chmod-socket = 666
vacuum = true

die-on-term = true

uwsgi log without gevent

your processes number limit is 15220
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on 0.0.0.0:5000 fd 3
uwsgi socket 0 bound to UNIX address PloutosBackend.sock fd 6
Python version: 3.8.0 (default, Oct 28 2019, 16:14:01)  [GCC 8.3.0]
PEP 405 virtualenv detected: /home/carlhung/uwsgienv
Set PythonHome to /home/carlhung/uwsgienv
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x55fb8d1761e0
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 364600 bytes (356 KB) for 4 cores
*** Operational MODE: preforking ***
Server initialized for gevent_uwsgi.
WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x55fb8d1761e0 pid: 2021 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 2021)
spawned uWSGI worker 1 (pid: 2034, cores: 1)
spawned uWSGI worker 2 (pid: 2035, cores: 1)
spawned uWSGI worker 3 (pid: 2036, cores: 1)
spawned uWSGI worker 4 (pid: 2037, cores: 1)
spawned uWSGI http 1 (pid: 2038)

uwsgi log with gevent

your processes number limit is 15220
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
- async cores set to 1000 - fd table size: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on 0.0.0.0:5000 fd 3
uwsgi socket 0 bound to UNIX address PloutosBackend.sock fd 6
Python version: 3.8.0 (default, Oct 28 2019, 16:14:01)  [GCC 8.3.0]
PEP 405 virtualenv detected: /home/carlhung/uwsgienv
Set PythonHome to /home/carlhung/uwsgienv
Python main interpreter initialized at 0x559dbd3b51f0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 52592320 bytes (51359 KB) for 4000 cores
*** Operational MODE: preforking+async ***
Server initialized for gevent_uwsgi.
WSGI app 0 (mountpoint='') ready in 4 seconds on interpreter 0x559dbd3b51f0 pid: 2074 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 2074)
spawned uWSGI worker 1 (pid: 2077, cores: 1000)
spawned uWSGI worker 2 (pid: 2078, cores: 1000)
spawned uWSGI worker 3 (pid: 2080, cores: 1000)
spawned uWSGI worker 4 (pid: 2081, cores: 1000)
spawned uWSGI http 1 (pid: 2082)
*** running gevent loop engine [addr:0x559dbc63b9f0] ***
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable

[miguelgrinberg]

I'm not sure, but given that you are not following the documentation on how to run uWSGI I suggest you start from the options that are known to work and then experiment with changes.

One thing that is 100% wrong is the workers. You should run one worker, that's the only thing that works.

Documentation: https://flask-socketio.readthedocs.io/en/latest/#uwsgi-web-server

[Pang654]

I have built a simple flask_socketio to test it, and I have followed the documentation and used the exact same command, but I still get the same error.

from gevent import monkey
monkey.patch_all()

from flask import Flask , jsonify
from flask_socketio import SocketIO, emit, join_room, leave_room, send, ConnectionRefusedError

app = Flask(__name__)
app.config['SECRET_KEY'] = "abc"
app_plain_text = "abc"

socketio = SocketIO(app, cors_allowed_origins="*", logger=True, engineio_logger=True)

app.config['DEBUG'] = True

def ack():
  print('message was received!')

# testing use only

@app.route("/test", methods=["POST"])
def test():
  return jsonify({"result": "successful"})

@socketio.on('connect')
def test_connect():
  print('Client connected')

@socketio.on('disconnect')
def test_disconnect():
  print('Client disconnected')

if __name__ == "__main__":
  socketio.run(app, host='0.0.0.0', port=5000)

uwsgi --http :5000 --gevent 1000 --http-websockets --master --wsgi-file server.py --callable app

*** Starting uWSGI 2.0.18 (64bit) on [Sat Apr 18 12:14:19 2020] ***
compiled with version: 7.5.0 on 17 April 2020 11:30:51
os: Linux-4.15.0-72-generic #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019
nodename: computer-Stick
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /home/carlhung/flask_socketio
detected binary path: /home/carlhung/uwsgienv/bin/uwsgi
your processes number limit is 15220
your memory page size is 4096 bytes
detected max file descriptor number: 1048576
- async cores set to 1000 - fd table size: 1048576
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :5000 fd 4
uwsgi socket 0 bound to TCP address 127.0.0.1:34275 (port auto-assigned) fd 3
Python version: 3.8.0 (default, Oct 28 2019, 16:14:01)  [GCC 8.3.0]
Python main interpreter initialized at 0x5626055208e0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 21036928 bytes (20543 KB) for 1000 cores
*** Operational MODE: async ***
Server initialized for gevent_uwsgi.
WSGI app 0 (mountpoint='') ready in 2 seconds on interpreter 0x5626055208e0 pid: 15132 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 15132)
spawned uWSGI worker 1 (pid: 15134, cores: 1000)
spawned uWSGI http 1 (pid: 15135)
*** running gevent loop engine [addr:0x562603e609f0] ***
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable
TypeError: 'module' object is not callable

Package List

amqp==2.5.2
bcrypt==3.1.7
beautifulsoup4==4.9.0
billiard==3.6.3.0
boto3==1.12.40
botocore==1.15.40
celery==4.4.2
certifi==2020.4.5.1
cffi==1.14.0
chardet==3.0.4
click==7.1.1
cryptography==2.9
dnspython==1.16.0
docutils==0.15.2
Flask==1.1.2
Flask-Bcrypt==0.7.1
Flask-SocketIO==4.2.1
gevent==1.5.0
greenlet==0.4.15
idna==2.9
itsdangerous==1.1.0
Jinja2==2.11.2
jmespath==0.9.5
kombu==4.6.8
MarkupSafe==1.1.1
monotonic==1.5
mysql-connector-python==8.0.19
pkg-resources==0.0.0
protobuf==3.6.1
pycparser==2.20
pycryptodomex==3.9.7
pyscrypt==1.6.2
python-dateutil==2.8.1
python-engineio==3.12.1
python-socketio==4.5.1
pytz==2019.3
redis==3.4.1
requests==2.23.0
s3transfer==0.3.3
selenium==3.141.0
six==1.14.0
soupsieve==2.0
urllib3==1.25.9
uWSGI==2.0.18
vine==1.3.0
Werkzeug==1.0.1

[miguelgrinberg]

Hmm. This happens with Python 3.8. but not with 3.7. So definitely not your or my issue.

[Pang654]

I switch it form uwsgi to gunicorn now, I think the matter wont be fixed soon

[johaven]

use gevent == 1.4.0 :)

[JadCham]

@Pang654 the issue is in uWSGI. It's been fixed part of https://github.com/unbit/uwsgi/pull/2149 .

[johaven]

in master but not released