Flask: No more than 5XX Requests/Second possible

[gobijan]

Hi.
I wrote a post about benchmarking some web frameworks and found a strange bug in Flask.
Requests per seconds were very low and not above 600 req/s. @miguelgrinberg contacted me about it and first thought that the issue was on my side but he also experienced the same behaviour.

Here is the Link to the Blogpost:
https://medium.com/@tschundeee/express-vs-flask-vs-go-acc0879c2122

Something might be blocking requests in Flask or Werkzeug.

I am willing to help towards inverstigation.
Cheers Bijan

[untitaker]

The dev server is not suitable for production. In fact it can only serve _one connection_ at a time. This has been stressed multiple times, both in IRC and the docs (at least in master, the quickstart contains a fairly explicit hint).

[gobijan]

I did not use the dev server. I used gunicorn...

[untitaker]

I didn't see that in your blogpost, only the app.run line.

In that case it'd be useful to know the exact gunicorn config.

On 23 June 2015 12:59:21 CEST, Bijan [email protected] wrote:

I did not use the dev server. I used gunicorn...

---

Reply to this email directly or view it on GitHub:
https://github.com/mitsuhiko/flask/issues/1512#issuecomment-114444837

[gobijan]

I just checked again on Python 2.7.10 (default, May 26 2015, 13:08:10)

I made it run via

gunicorn -w 8 flask:app

I started the benchmark via

wrk -c 64 -d 30s http://localhost:8000/10

My gunicorn installation was just installed via pip on osx 10.9.5 (I did not change anything):

The flask source can be found here:
https://github.com/tschundeee/fibonacci-web-lang-benchmark/blob/master/python/flask.py

[gobijan]

Could you try it on your setup and let me know if you experience the same behaviour?

PS: In case you wonder: I used https://github.com/wg/wrk as the benchmark tool because it is not as limited as ab.

[methane]

Try meinheld.

$ pip install meinheld
$ gunicorn -w8 flask:app -k meinheld.gmeinheld.MeinheldWorker

gunicorn's default worker is not designed for high load.
Since it doesn't support keep-alive, client connect to gunicorn for each request.
When closing TCP connection, it is in TIME_WAIT state for 1 minute.
If your machine uses 30000 free port, you can only 30000 req/min (= 500 req/sec).

I use nginx + uWSGI (reverse proxy via unix domain socket) for production and
use gunicorn/Meinheld (supports keep-alive) for benchmark.

[gobijan]

Nice!

brmpb:app bijan$ wrk -c 64 -d 30s http://localhost:8000/10
Running 30s test @ http://localhost:8000/10
2 threads and 64 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 6.02ms 4.92ms 72.68ms 76.25%
Req/Sec 5.90k 1.11k 8.26k 68.33%
352591 requests in 30.00s, 64.90MB read
Requests/sec: 11751.58
Transfer/sec: 2.16MB

[gobijan]

This deployment option should be added to the docs under:
http://flask.pocoo.org/docs/0.10/deploying/wsgi-standalone/

[iabtyagi]

@methane

"I use nginx + uWSGI (reverse proxy via unix domain socket) for production and
use gunicorn/Meinheld (supports keep-alive) for benchmark."

I am new to Flask and was exploring various deployment options. Can you please explain why you wouldn't use gunicorn/Meinheld for production? Is your comment still relevant?

[methane]

Because infra team prefer daemon completely written in C.
You can use gunicorn to listen unix domain socket.

[Valve]

@methane

gunicorn's default worker is not designed for high load.
Since it doesn't support keep-alive, client connect to gunicorn for each request.
When closing TCP connection, it is in TIME_WAIT state for 1 minute.
If your machine uses 30000 free port, you can only 30000 req/min (= 500 req/sec).

Can you please let me know where to read more about it? I would like to understand more about this, but this is a hard to formulate question

[methane]

Google "TIME_WAIT" and learn TCP states.

[Valve]

thanks @methane ! This is sufficient to get me started