Uvloop: Looks like loop is blocking with aiohttp.worker.GunicornWebWorker

[f0t0n]

When I deploy an aiohttp app with aiohttp.worker.GunicornWebWorker it looks like the loop is blocked and app never serves requests.

When I run without the worker (with web.run_app(app)), it works well.
I think the problem could be caused by the GunicornWebWorker itself since it's closing the default asyncio's loop and then creating new one. But I'm not sure.

Is this issue related to uvloop or I should post to aiohttp's issue tracker?

[1st1]

Maybe it's a bug in uvloop -- looks like Gunicorn is forking a running event loop, and then closing it (which is a very bad way of doing that). I'll take a look.

[f0t0n]

So the example basic app could look like:

import asyncio
import uvloop
from aiohttp import web


def index(request):
    return web.Response(text='Welcome home!')


def get_loop():
    asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
    return asyncio.get_event_loop()


app = web.Application(loop=get_loop())
app.router.add_route('GET', '/', index)

web.run_app(app, port=8888)

It works only without GunicornWebWorker.

[1st1]

So I copied-pasted your example app code in a test.py file. I then tried the following command:

gunicorn test:app --worker-class aiohttp.worker.GunicornWebWorker --bind localhost:8080

and it worked (on Mac OS & Linux).

Then I tried to launch it with a few workers:

gunicorn test:app --worker-class aiohttp.worker.GunicornWebWorker --workers 5 --bind localhost:8080

This time, it failed with "address already in use". I then tried to run it with asyncio event loop, and it also errored out, albeit in a different way.

Anyways, how exactly are you running your program, what exactly does it print to stderr/stdout, what's your OS, Python, uvloop/gunicorn/aiohttp versions?

[f0t0n]

@1st1 did you run my example with gunicorn with commenting/removing
the last line web.run_app(app, port=8888)?

This example is to run without Gunicorn, and it works well:

import asyncio
import uvloop
from aiohttp import web


def index(request):
    return web.Response(text='Welcome home!')


def get_loop():
    asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
    return asyncio.get_event_loop()


app = web.Application(loop=get_loop())
app.router.add_route('GET', '/', index)

web.run_app(app, port=8888)

Another one (without the last line) to run by Gunicorn is not working:

import asyncio
import uvloop
from aiohttp import web


def index(request):
    return web.Response(text='Welcome home!')


def get_loop():
    asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
    return asyncio.get_event_loop()


app = web.Application(loop=get_loop())
app.router.add_route('GET', '/', index)

# web.run_app(app, port=8888)

When I start with gunicorn it just hangs like listed below, and doesn't serve requests.

$ gunicorn test:app --worker-class aiohttp.worker.GunicornWebWorker --bind localhost:8888
[2016-05-17 00:46:17 +0000] [84] [INFO] Starting gunicorn 19.4.5
[2016-05-17 00:46:17 +0000] [84] [INFO] Listening at: http://127.0.0.1:8888 (84)
[2016-05-17 00:46:17 +0000] [84] [INFO] Using worker: aiohttp.worker.GunicornWebWorker
[2016-05-17 00:46:17 +0000] [99] [INFO] Booting worker with pid: 99

OS:

~ $ cat /etc/*-release
3.3.0
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.3.0
PRETTY_NAME="Alpine Linux v3.3"
HOME_URL="http://alpinelinux.org"
BUG_REPORT_URL="http://bugs.alpinelinux.org"

Python:

~ $ python --version
Python 3.5.1

Gunicorn:

~ $ gunicorn --version
gunicorn (version 19.4.5)

uvloop:

~ $ pip freeze | grep uvloop
uvloop==0.4.24

aiohttp:

~ $ pip freeze | grep aiohttp
aiohttp==0.21.5

[1st1]

Alright, I got it now -- will debug.

[1st1]

@f0t0n I think I've figured it out. It seems that it's too late to set up a uvloop policy in your app -- you have to do that at the gunicorn worker level. I've created a PR for the aiohttp project.

[ssbb]

Both PR are merged and looks like issue is fixed. But only worker class should be specified w/o installing event loop policy manually.

[machbio]

What version will this fix be available ?

[1st1]

@asvetlov Andrew, when will you make a new aiohttp release?

[1st1]

aiohttp 0.22 was released a few days ago. Closing this one now, thanks everybody!

[machbio]

Upgraded but still not working - requests are still blocked and hangs -

uvloop==0.5.0
aiohttp==0.22.3

@1st1 what do you mean by
"too late to set up a uvloop policy in your app -- you have to do that at the gunicorn worker level"

[1st1]

@machbio Please see http://aiohttp.readthedocs.io/en/stable/gunicorn.html#start-gunicorn, specifically the note about uvloop.

[1st1]

@1st1 what do you mean by
"too late to set up a uvloop policy in your app -- you have to do that at the gunicorn worker level"

It literally means that in order to use uvloop with aiohttp+gunicorn you have to use a special gunicorn worker class that aiohttp now bundles:

$ gunicorn ... --worker-class aiohttp.worker.GunicornUVLoopWebWorker

And you shouldn't really do anything else in your app.

[machbio]

Thank you @1st1 - the above solution works