Gunicorn: Understanding Gunicorn Sync Workers working for handling multiple requests

[saurabhprakash]

I am using Sync Workers of Gunicorn with 4 workers running.. Lets say 100 request hit the server at same time.. How does gunicorn handle this, Does all 4 workers have 25 requests each Or master worker maintains the queue and sends the requests to child worker based on their availability ?

[benoitc]

@saurabhprakash http://docs.gunicorn.org/en/stable/design.html

The worker doesn't maintain any queue. Everything arrive in the backlog of the listener, the system balance the connection to the processes that wait on accept.

[benoitc]

closing the issue, response have been done.

[krishnand]

Can you clarify your statement above? What does the "backlog of the listener" mean here? Taking the same example quoted above, did you mean to say all the requests would be queued on the client itself until any worker is ready to accept a new request?

[tilgovi]

@krishnand http://veithen.io/2014/01/01/how-tcp-backlog-works-in-linux.html

[tilgovi]

You can control the value that Gunicorn provides to the operating system with the backlog setting: http://docs.gunicorn.org/en/latest/settings.html#backlog

[sashasimkin]

What is the max time that single connection will spend in backlog queue?

I.e. the same situation as in the first post, but the gunicorn timeout is set to 90, and those 4 requests take all 90s to complete - will remaining 96 requests be processed?

[tilgovi]

@sashasimkin they will spend as long as it takes to get to them, unless the operating system receives a packet from the remote client that shuts down the socket.

[tilgovi]

It may help to refer to a TCP state diagram. The backlog contains connections in the ESTABLISHED state. As long as the connection remains ESTABLISHED, Gunicorn will eventually accept() them.

How long it takes to get them is bounded by the timeout, but it could be much quicker if requests are handled quickly.

http://users.cs.northwestern.edu/~agupta/cs340/project2/TCPIP_State_Transition_Diagram.pdf

[tilgovi]

Here's another useful read: https://blog.cloudflare.com/syn-packet-handling-in-the-wild/

[sashasimkin]

Thanks, I'll read through the links in detail and try to understand what's going on, tho my situation looks slightly differently:

I'm using artillery to load-test the application(sync worker), and at some point requests do timeout after 60 seconds, while both nginx(proxy read/connect) and gunicorn timeout have value 90s.

In addition - in the report I get numbers like this, while every single request takes 3s to 40s to be processed (according to metrics sent into statsd)

Elapsed time: 9 minutes, 10 seconds
  Scenarios launched:  259
  Scenarios completed: 57
  Requests completed:  736
  RPS sent: 71.4
  Request latency:
    min: 0.9
    max: 60054.7
    median: 8
    p95: 60011.2
    p99: 60016.7
  Codes:
    200: 160
    500: 370
    502: 22
    504: 184

What I'm trying to understand is - where the number 60 comes from? (while with gthread worker - max timeouts were 90s).

Also, during the peak load there are 453 established sockets and constantly 0 in syn-recv state.

---

UPD: It was simple: 60s is LB's timeout :( sorry for this :(

[tilgovi]

UPD: It was simple: 60s is LB's timeout :( sorry for this :(

No problem. Thanks for following up. I hope the links are useful, nonetheless, and that I wasn't just telling you things you already know. 😄

[sashasimkin]

Sincerely, I knew _some_ of the things there, yet definitely missing some deeper points and those links are of great use there, thanks again!

Btw, I also found the reason of HTTP 500 errors, which is also quite simple: [alert] 892#892: *50315 768 worker_connections are not enough while connecting to upstream 🤷‍♂