Flask-socketio: Why can Gunicorn servers serving socketio only use one worker? What is the effect of this on performance?

[ghost]

In the deployment section of the docs, you write that:

Due to the limited load balancing algorithm used by gunicorn, it is not possible to use more than one worker process when using this web server. For that reason, all the examples above include the -w 1 option.

What specific problem of the load balancing algorithm is preventing this? Will this limit the performance of applications using Gunicorn and flask-socketio?

[miguelgrinberg]

Gunicorn does not support sticky sessions. They were thinking about adding it at some point, but until they do, you can only have one worker.

But this isn't terrible, because you can use nginx as a load balancer, running several gunicorn process, all with one worker.

[ghost]

Thanks for the answer. We went with several Gunicorn instances load-balanced by Nginx. I guess I'll have to start pressuring Gunicorn to fix their mess.

[nmvega]

Ah, this may be my issue!

Days ago I installed nginx + gunicorn (1-unix process with 5-gunicorn workers) to run a Django forum app; and I couldn't figure out, after editing a test post, why refreshing the browser would toggle/flap between pre and post edited content (even when I cleared the browser caches). Nothing I tried over many days worked.

As a last resort last night, I set the number of gunicorn workers to 1, which appears to have fixed the issue. I then went in search for answers today, and found this thread. o/

My apologies in advance for these next questions because, while technical, I'm not a web developer nor web master. (._.) But assuming that this is my issue (feel free to confirm), and that I have to "separate" connections over, say, 5, 1-worker gunicorn unix processes load-balanced by nginx, do I:

1. Start 5, 1-worker gunicorn unix processes pointing to the same wsgi.py file, and have each bind to a different socket? (.../myapp.sock1, ..., .../myapp.sock5). Or can/should the same socket be used? _SideNote: Using the same socket could be risky if, say, this, sudo supervisorctl stop gunicorn.instance1, tries to delete it._

2. Isolating the following two sections of mynginx.conf file (i.e. the non static sections), how should each section be adjusted to feed those five (qty. 5) gunicorn instances above (whether or not they use the same Unix domain socket).

Sorry for the long question. I actually posted this question on StackOverflow (before I figured out that this may be my issue, so that question was way longer). So thank you in advance for reading and sharing!

upstream myapp_server {
    server unix:/var/run/myapp.d/myapp.sock fail_timeout=0;
    # =============================================================
    # Path to DYNAMIC (NOT STATIC) content (when using GUNICORN).
    # =============================================================
}

---

# ===========================================
# Location block for DYNAMIC CONTENT served by GUNICORN ...
# ===========================================
location / {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://myapp_server;
    proxy_buffering off;
}
#

[miguelgrinberg]

You can start 5 single-worker gunicorns, each on its own port (or unix socket). In the upstream section add the iphash option to enable sticky sessions (if not you would have the same problem as before), and then write five server lines pointing to your gunicorn instances. The location section looks good as you have it, I think.

[nmvega]

Wow! That was too fast a response Miguel. =:) Thank you. I think I understand your recommendations. Let me try them and I will give feedback to you and friends in this thread. Thank you again Miguel.

[nmvega]

I'm starting to refactor the gunicorn start/stop script as well as nginx.conf now, but have a super practical decision to make. I'm currently doing this on one VPS, but thinking of a two-tier approach (for better security, and to have a second server to back up to), like so:

Either:

- On vps01: nginx-on-vps01 -- Points to vps02:80 --> nginx-on-vps02
- On vps02: nginx-on-vps02 -- Upstreams to UNIX Domain Sockets --> gunicorn(s)-on-vps02

-- Or (a little simpler):

- On vps01: nginx-on-vps01 <-- Points to vps02-IP; and multiple TCP-ports --> gunicorn(s)-on-vps02
- On vps02: gunicorn(s) on vps02-IP, running on multiple TCP-ports.

For either case, I would probably also employ nginx directives and firewalld rules (as a protective measure) to only allow traffic over the two vps-IP/PORT combinations, to talk to each other.

The second option seems a little easier (complexity and, later on, troubleshooting wise); but the first one seems more stout. Is one more preferable or more well-traveled?

P.S. I'll figure out either way, one or the other. :) Thanks.

[miguelgrinberg]

@nmvega If I had two VPSs at my disposal I would split my five workers among them. Then you can run nginx on both, configured to forward traffic to the five gunicorns that are distributed among the two servers. With this set up you have two entry points to your application, and if any of the two servers go down, you can still handle end-to-end requests on the other one.

[nmvega]

@miguelgrinberg That's an even better idea. I know better, and should have thought Active/Active from the start (... I'm too in the weeds right now, unfortunately, so thank you for that).

Note: _Actually I meant six instances of gunicorn, because there are 6-vCPUs (so 3 + 3)_.

So, assuming you don't mean nginx to nginx communication anywhere but, rather, nginx to local & remote instances of gunicorn directly, what you described -- I believe -- looks like this in the upstream stanzas of the respective nginx.conf files:

nginx.conf on vps01:

upstream myapp_server {
    ip_hash; # Needed for my original dynamic content issue.  <------

    # Prefer the local server using relative weights ...
    server vsp01.example.com:8081 weight=20 max_fails=3 fail_timeout=30s;
    server vsp01.example.com:8082 weight=20 max_fails=3 fail_timeout=30s;
    server vsp01.example.com:8083 weight=20 max_fails=3 fail_timeout=30s ;

    server vsp02.example.com:8081 weight=10 max_fails=3 fail_timeout=30s;
    server vsp02.example.com:8082 weight=10 max_fails=3 fail_timeout=30s;
    server vsp02.example.com:8083 weight=10 max_fails=3 fail_timeout=30s;
}

nginx.conf on vps02 (which is essentially the inverse of the above):

upstream myapp_server {
    ip_hash; # Needed for my original dynamic content issue.  <------

    # Prefer the local server using relative weights ...
    server vsp02.example.com:8081 weight=20 max_fails=3 fail_timeout=30s;
    server vsp02.example.com:8082 weight=20 max_fails=3 fail_timeout=30s;
    server vsp02.example.com:8083 weight=20 max_fails=3 fail_timeout=30s ;

    server vsp01.example.com:8081 weight=10 max_fails=3 fail_timeout=30s;
    server vsp01.example.com:8082 weight=10 max_fails=3 fail_timeout=30s;
    server vsp01.example.com:8083 weight=10 max_fails=3 fail_timeout=30s;
}

Summary (from a non webmaster (◠﹏◠) ) :

• nginx to local & remote gunicorns instances directly. (No nginx -to-> ngin -to-> gunicorn).
• Setup Round-Robin DNS on the example.com NS hoster, pointing to both vps01/02.example.com
• I have SSL certificates only for example.com and www.example.com. Hmm... Will I need them for vps01/02.example.com on the backend?

Thank you!

[nmvega]

Just an update that, at least on one vps, vps01 (because I haven't yet rented another), this thread resolved my critical, show-stopping, dynamic-content issue: By launching 6, 1-worker gunicorn UNIX processes and adding the ip_hash directive to the upstream stanza of nginx.conf, things went from being nearly 100% unstable (flapping dynamic content and postgres errors), to 100% stable with no errors. Thank God! I nearly tore my eyes out the last few days. ¯\__(⊙︿⊙)__/¯ Thank you again Miguel.

Noelle

[pwnbase]

Just an update that, at least on one vps, vps01 (because I haven't yet rented another), this thread resolved my critical, show-stopping, dynamic-content issue: By launching 6, 1-worker gunicorn UNIX processes and adding the ip_hash directive to the upstream stanza of nginx.conf, things went from being nearly 100% unstable (flapping dynamic content and postgres errors), to 100% stable with no errors. Thank God! I nearly tore my eyes out the last few days. ¯*(⊙︿⊙)*/¯ Thank you again Miguel.

Noelle

Hi @nmvega, I've tested this solution and it works, just wondering if anyone's come up with a better gunicorn multiple worker config?

[miguelgrinberg]

@pwnbase Because of the way Gunicorn does load balancing it is unlikely it will ever support sticky sessions, which are required for the long-polling part of the Socket.IO protocol to work. So no, I do not expect a multi-worker setup with gunicorn to work any time soon.