Gevent: Gevent pywsgi WSGIServer concurrent request are queued

[kleysonr]

• gevent version: 1.4.0
• Python version: 3.6
• Operating System: Ubuntu 18.04.2 LTS

Description:

I'm using Connexion with Gevent WSGI server to create an API server.
My API is CPU bound.

Concurrent calls to the API are being executed in a serial fashion.

Below the start/end time of the API execution:

2019-06-07 10:57:43,219 - ssp.ws.rgvirtual - INFO - Starting api/rgf for requestid 1
2019-06-07 10:57:55,475 - ssp.ws.rgvirtual - INFO - api/rgf ran in 12.26s for requestid 1
2019-06-07 10:57:55,487 - app.main - INFO - 10.x.x.x - - [2019-06-07 10:57:55] "POST /api/rgf?requestid=1&width=1024&widthret=200&rotate=0 HTTP/1.1" 200 84199 12.438735

2019-06-07 10:57:55,511 - ssp.ws.rgvirtual - INFO - Starting api/rgv for requestid 2
2019-06-07 10:58:07,217 - ssp.ws.rgvirtual - INFO - api/rgv ran in 11.71s for requestid 2
2019-06-07 10:58:07,282 - app.main - INFO - 10.x.x.x - - [2019-06-07 10:58:07] "POST /api/rgv?requestid=1&width=1024&widthret=500&rotate=0 HTTP/1.1" 200 541706 24.228940

What I've run:

This is how the WSGIServer is created (https://github.com/zalando/connexion/blob/2.0.2/connexion/apps/flask_app.py#L106-L113):

        elif self.server == 'gevent':
            try:
                import gevent.pywsgi
            except ImportError:
                raise Exception('gevent library not installed')
            http_server = gevent.pywsgi.WSGIServer((self.host, self.port), self.app, **options)
            logger.info('Listening on %s:%s..', self.host, self.port)
            http_server.serve_forever()

What is needed to have my CPU bound requests be processed in parallel by the Gevent ?
What are the difference between 'wsgi.multithread' and 'wsgi.multiprocess' ?

I tried to enable the 'wsgi.multithread' and 'wsgi.multiprocess' parameters but I got the same behavior:

geventOpt = {'GATEWAY_INTERFACE': 'CGI/1.1',
                'SERVER_SOFTWARE': 'gevent/%d.%d Python/%d.%d' % (gevent.version_info[:2] + sys.version_info[:2]),
                'SCRIPT_NAME': '',
                'wsgi.version': (1, 0),
                'wsgi.multithread': True, # XXX: Aren't we really, though?
                'wsgi.multiprocess': True,
                'wsgi.run_once': False}
http_server = gevent.pywsgi.WSGIServer((self.host, self.port), self.app, environ=geventOpt)

[jamadden]

gevent is for cooperative multitasking: if your code is CPU bound and not cooperative, there's no automatic way for gevent to extract concurrency. You can try running your CPU bound code in a threadpool to use multiple threads and extract some level of concurrency and parallelism that way (limited to Python's GIL). (E.g., result = gevent.get_hub().threadpool.apply(cpu_func, (arg1, arg2))). Or you can insert calls to gevent.sleep(0) into your inner loop to make the loop cooperative.

[kleysonr]

@jamadden I saw some examples that it's supposed to work.

For the testing purposes, my func() is basically a time.sleep(10).

Using a sleep function for testing, should the gevent be able to process the next request before finishing the first one ?

[jamadden]

It depends on the details. If it's actually time.sleep(10) then unless the system is monkey-patched by gevent the answer is an emphatic "no!". time.sleep is definitely not cooperative.

[kleysonr]

Applying monkey_patch I am able to process the requests in parallel.

from gevent import monkey
monkey.patch_all()

Thanks.