Uwsgi: Make pyuwsgi plugin python 3 compatible

[msurdi]

[unbit]

Do you use it or you are a packager ? I am asking because we were thinking about removing it, but if you have a use it for it we can start mantaining it again. Thanks

[msurdi]

I was just looking for a way to run uwsgi from a freezed (cx_freeze) application. TBH, this is just some experimentation so I'm not using it right now, just investigating if this could work in this setup so I can replace gunicorn.

[anthonyrisinger]

I am using this. Combined with setup.cpyext.py in 2.1 you get a real python module, uswgi.so, and you can call uwsgi.setup return to python and do more work (in the master while interacting with uWSGI api) then call uwsgi.run.

IMO this should be the default for pip installations at the least, and fixes at least 2-3 pip related bugs in the tracker.

[funkybob]

Is this issue still outstanding, or has it been fixed?

[natefoo]

@unbit any chance this will see some work? If not, I have looked at it a bit, and can commit some more effort to it in a few months.

We do use pyuwsgi quite heavily, as it is (to my knowledge) the only way to provide a precompiled uwsgi that does not either: 1. require a dynamic libpython to be installed on the system, or 2. statically link libpython. This is how we build uWSGI wheels for wheels.galaxyproject.org and aside from a few non-showstopping bugs, it works well for Python 2.

[ipmb]

For anybody keeping an eye on this issue (and wanting to build wheels), Lincoln Loop is sponsoring development for it. @unbit just committed a fix for building pyuwsgi on Python 3. From the uwsgi-2.0 branch, the shared library can be built with:

UWSGI_EMBED_PLUGINS=python make pyuwsgi PYTHON=python3

but I'm hoping we can get to the point where that just happens as part of the pip install. 🤞

[natefoo]

Thanks for your sponsorship @ipmb! The changes in #1863 make a pip-installable wheel if built with python setup.cpyext.py bdist_wheel.

At present, there's no way to specify at pip install time whether you want standard uWSGI or pyuwsgi. The only way I can think to do this for PyPI is to only distribute the sdist under the uWSGI project, and place wheels under a separate "pyuwsgi" project (@unbit you may want to grab pyuwsgi in PyPI if this path is of interest to you), and then instruct people to use pip install pywusgi for a precompiled uWSGI.

We build pyuwsgi wheels as uWSGI (not pyuwsgi) because we're making the explicit decision to give people pyuwsgi if they point pip install uwsgi at our index server (which still allows falling back to PyPI if a wheel for their architecture is not available), but I assume this is not desirable for the official source in PyPI.

[ipmb]

I published a preliminary sdist to https://pypi.org/project/pyuwsgi/

I'm hoping to get manylinux wheels published soon and am working on that here: https://github.com/lincolnloop/pyuwsgi-wheels.

Once everything is working, I'll remove the pre-release tag from PyPI and hopefully just follow step with uWSGI's release cycle.

[natefoo]

@ipmb Great, this is better than my efforts so far. Two potential outstanding issues with wheels in PyPI (that maybe should get their own issue(s) but I'm mentioning them here since you're currently working on the packaging):

1. There's not a great way with the standard PyPA tooling to provide wheels with differing functionality. Some, such as linking libpcre, is probably safe to include in all wheels since it only enables additional functionality but does not otherwise change the behavior of a non-PCRE build. But others, such as linking libyaml (#863), change the normal behavior in incompatible ways. To support both, uWSGI needs a switch to specify "use internal YAML parsing even if built with libyaml," but maybe there is some setuptools extras_require trick that can be used here (e.g. for pip install pyuwsgi[libyaml]).
2. It's not possible to bundle nonstandard libs into the macOS wheel currently (libyaml, libpcre, ...) due to a limitation in delocate described in matthew-brett/delocate#12 (comment) unless the pyuwsgi shared lib is placed inside a dummy package. It may make more sense to fix delocate rather than pyuwsgi, however.

[ipmb]

Thanks for the feedback @natefoo. I'm definitely fumbling my way through some of the packaging stuff, so your feedback is appreciated.

There's not a great way with the standard PyPA tooling to provide wheels with differing functionality.

I'm leaning towards the wheel configuration being opinionated and if you want something different, you can always build your own from the sdist. My current thinking is to bundle zlib, pcre, libyaml, and jansson, but exclude ssl. Would that work for your use case?

It's not possible to bundle nonstandard libs into the macOS wheel currently (libyaml, libpcre, ...)

Shoot! I didn't realize that. I'm open to any ideas, that's beyond my area of expertise :)

Could you try out the latest wheels I uploaded and see how they work for you?

pip install -U --pre pyuwsgi

It might make sense to move discussion on the wheels to https://github.com/lincolnloop/pyuwsgi-wheels

[natefoo]

Thanks, I moved the other issues over to lincolnloop/pyuwsgi-wheels#1 and lincolnloop/pyuwsgi-wheels#2.

I tested two - both pyuwsgi-2.0.17.2a4-cp27-cp27mu-manylinux1_x86_64.whl and pyuwsgi-2.0.17.2a4-cp35-cp35m-manylinux1_x86_64.whl work for me.

[ipmb]

This issue can be closed. pyuwsgi is available on PyPI and Python 3 compatible.

[xrmx]

Thanks for the update