uvloop build failed on Python 3.8

[Snawoot]

• uvloop version: 0.13.0
• Python version: 3.8
• Platform: Linux (Ubuntu Xenial, Fedora 30)
• Can you reproduce the bug with PYTHONASYNCIODEBUG in env?: No, can't even build uvloop

Hello,

Two days ago new stable Python version 3.8.0 has been released. Today I updated test matrix for one of my projects and noticed uvloop fails to install: https://travis-ci.org/Snawoot/postfix-mta-sts-resolver/jobs/598694743

Steps to reproduce:

pip3.8 install uvloop

[sonirico]

Same here, same platform, uvloop and python version: https://travis-ci.com/sonirico/wpoke/jobs/246022440
This is a big stopper in regards to make libraries take advantage of the new features introduced in py38. Fast feedback on this is much appreciated...

[Jamim]

It looks like this issue duplicates #266.

[Snawoot]

@Jamim Yes, it is, but this issue also escalates problem. Now uvloop is not usable on current stable python.

[speedyb0y]

Just a feedback: sucesfully built uvloop with Python 3.8, Cython 0.29.13. Using with latest aiohttp and websockets.

Have 6 processes running for 4 hours, dealing with gigabytes of data, with a total of ~8000 connections.
Everything seems fine.

As others said the only change needed was to use Cython 0.29.13 on uvloop.pyx.

[1st1]

Quick update: I'll try to find time to release a new version in a couple of days. Thanks for confirming that Cython 0.29.13 works.

[ndaidong]

@speedyb0y what did you do exactly? Within virtual environment created with python 3.8, I tried to install cython 0.29.3 first, then install uvloop. It sill fails.

Installing collected packages: uvloop
    Running setup.py install for uvloop ... error
    ERROR: Command errored out with exit status 1:
     command: /workspace/awl/abms-v2/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-2ej5cuaf/uvloop/setup.py'"'"'; __file__='"'"'/tmp/pip-install-2ej5cuaf/uvloop/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-yawpg18_/install-record.txt --single-version-externally-managed --compile --install-headers /workspace/awl/abms-v2/venv/include/site/python3.8/uvloop
         cwd: /tmp/pip-install-2ej5cuaf/uvloop/
    Complete output (1175 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.8
    creating build/lib.linux-x86_64-3.8/uvloop
    copying uvloop/_noop.py -> build/lib.linux-x86_64-3.8/uvloop
################################################################
# stripped....
################################################################
      CC       src/unix/libuv_la-procfs-exepath.lo
      CC       src/unix/libuv_la-proctitle.lo
      CC       src/unix/libuv_la-sysinfo-loadavg.lo
      CCLD     libuv.la
    ar: `u' modifier ignored since `D' is the default (see `U')
    building 'uvloop.loop' extension
    creating build/temp.linux-x86_64-3.8
    creating build/temp.linux-x86_64-3.8/uvloop
    gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/workspace/awl/abms-v2/venv/include -I/usr/local/include/python3.8 -I/tmp/pip-install-2ej5cuaf/uvloop/vendor/libuv/include -c uvloop/loop.c -o build/temp.linux-x86_64-3.8/uvloop/loop.o -O2
    uvloop/loop.c: In function ‘__Pyx_InitCachedConstants’:
    /usr/local/include/python3.8/code.h:72:25: warning: passing argument 6 of ‘PyCode_New’ makes pointer from integer without a cast [-Wint-conversion]
     #define CO_OPTIMIZED    0x0001
                             ^
    uvloop/loop.c:334:37: note: in definition of macro ‘__Pyx_PyCode_New’
               PyCode_New(a, 0, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)
                                         ^
    uvloop/loop.c:154727:63: note: in expansion of macro ‘CO_OPTIMIZED’
       __pyx_codeobj__13 = (PyObject*)__Pyx_PyCode_New(1, 0, 3, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__12, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_uvloop_loop_pyx, __pyx_n_s_callback, 850, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__13)) __PYX_ERR(2, 850, __pyx_L1_error)
                                                                   ^~~~~~~~~~~~
    In file included from /usr/local/include/python3.8/compile.h:5,
                     from /usr/local/include/python3.8/Python.h:138,
                     from uvloop/loop.c:20:
    /usr/local/include/python3.8/code.h:122:28: note: expected ‘PyObject *’ {aka ‘struct _object *’} but argument is of type ‘int’
     PyAPI_FUNC(PyCodeObject *) PyCode_New(
                                ^~~~~~~~~~
    uvloop/loop.c:154727:230: warning: passing argument 14 of ‘PyCode_New’ makes integer from pointer without a cast [-Wint-conversion]
       __pyx_codeobj__13 = (PyObject*)__Pyx_PyCode_New(1, 0, 3, 0, CO_OPTIMIZED|CO_NEWLOCALS, __pyx_empty_bytes, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_tuple__12, __pyx_empty_tuple, __pyx_empty_tuple, __pyx_kp_s_uvloop_loop_pyx, __pyx_n_s_callback, 850, __pyx_empty_bytes); if (unlikely(!__pyx_codeobj__13)) __PYX_ERR(2, 850, __pyx_L1_error)
                                                                                                                                                                                                                                          ^~~~~~~~~~~~~~~~~~
    uvloop/loop.c:334:69: note: in definition of macro ‘__Pyx_PyCode_New’
               PyCode_New(a, 0, k, l, s, f, code, c, n, v, fv, cell, fn, name, fline, lnos)
                                                                         ^~~~
    In file included from /usr/local/include/python3.8/compile.h:5,
                     from /usr/local/include/python3.8/Python.h:138,
                     from uvloop/loop.c:20:
    /usr/local/include/python3.8/code.h:122:28: note: expected ‘int’ but argument is of type ‘PyObject *’ {aka ‘struct _object *’}
     PyAPI_FUNC(PyCodeObject *) PyCode_New(
                                ^~~~~~~~~~
    uvloop/loop.c:154727:250: warning: passing argument 15 of ‘PyCode_New’ makes pointer from integer without a cast [-Wint-conversion]

[imbolc]

@ndaidong I guess he used master's version:

pip install git+https://github.com/MagicStack/uvloop

[h1nk]

@ndaidong see this comment & insurgency/aioa2squery@9e63733

[speedyb0y]

hi @ndaidong , I use the BuildRoot project to build my entire system.
It seems to use "pip install" under the hood so I just downloaded the current version uvloop-VERSION.tar.gz, runned "cython -3 uvloop/loop.pyx" and it generated a new uvloop/loop.pyx.c file.

I then made a new tar.gz and used it instead of the original.

I used my distribution Cython, which is the latest version.

[ndaidong]

@imbolc @h1nk @speedyb0y : thank you all for your replies
I confirmed that uvloop from its master branch works for me. Just need to install latest version of Cython first, then run the command like imbolc's suggestion.
Another way that helps to install it faster is build uvloop first (similar to my fork), then use it later without Cython and rebuilding.

[shusso]

Just confirming that uvloop master + python3.8 seems to work (in the python3.8 docker image):

FROM python:3.8.0-slim-buster

ADD ./requirements.txt /

RUN apt-get update -y
RUN apt-get install -y gcc git build-essential libtool automake
RUN pip3 install Cython
RUN pip3 install git+https://github.com/MagicStack/uvloop
...

[1st1]

I do not recommend running uvloop 0.13 on 3.8. It is broken in many subtle ways. A proper release is coming soon, please wait.

[1st1]

I've just published https://github.com/MagicStack/uvloop/releases/tag/v0.14.0rc1. Please test. The final release will be published some time next week if RC1 is OK.

[Snawoot]

My test suite is passing. Thank you! Looking forward for PyPI release.

[1st1]

Thank you! Looking forward for PyPI release.

You can install with pip install uvloop==0.14.0rc1