Uwsgi: undefined symbol when using setproctitle with a manually compiled python

[azmeuk]

How to reproduce the bug

I just manually compiled python 2.7.15 on an up-to-date debian 9, with the default options.

./configure --enable-optimizations
make
sudo make altinstall

Then I created a virtualenv using the compiled python and installed uwsgi 2.0.17.1 and setproctitle.

virtualenv -p /usr/local/bin/python2.7 env
./env/bin/pip install --no-cache uwsgi setproctitle

With a minimalist application file that just imports setproctitle, uwsgi crashes.

echo "import setproctitle" > application.py
./env/bin/python application.py # nothing happens, everything goes fine
./env/bin/uwsgi --wsgi-file application.py --http :8005

```
* Starting uWSGI 2.0.17.1 (64bit) on [Wed Jul 18 13:49:37 2018]
compiled with version: 6.3.0 20170516 on 18 July 2018 09:54:02
os: Linux-4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07)
nodename: *
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 24
current working directory: /home/eloi
detected binary path: /home/eloi/env/bin/uwsgi
WARNING: you are running uWSGI without its master process manager
your processes number limit is 515314
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uWSGI http bound on :8005 fd 4
spawned uWSGI http 1 (pid: 454)
uwsgi socket 0 bound to TCP address 127.0.0.1:34625 (port auto-assigned) fd 3
Python version: 2.7.15 (default, Jul 17 2018, 12:03:44) [GCC 6.3.0 20170516]
Python threads support is disabled. You can enable it with --enable-threads
Python main interpreter initialized at 0x5620c788b5a0
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72920 bytes (71 KB) for 1 cores
Operational MODE: single process
Traceback (most recent call last):
File "application.py", line 1, in
import setproctitle
ImportError: /home/eloi/env/lib/python2.7/site-packages/setproctitle.so: undefined symbol: Py_GetArgcArgv
unable to load app 0 (mountpoint='') (callable not found or import error)
no app loaded. going in full dynamic mode
uWSGI is running in multiple interpreter mode *
spawned uWSGI worker 1 (and the only) (pid: 453, cores: 1)

# Question
So I am not sure what is wrong here. Is it about uwsgi, setproctitle or the way I compile python?

- As setproctitle with the compiled python and the compiled python has no other issue, I though maybe there is something with uwsgi and the way it deals with dynamic libraries?
- However, if I use the debian system packages python2 `/usr/bin/python2` (that is 2.7.13) to build my virtualenv, uwsgi does not crash.

Is there some special configuration I should use when building python from sources to allow uwsgi to correctly manage dynamic libraries?

# Extra information:
```bash
ldd env/bin/uwsgi
        linux-vdso.so.1 (0x00007ffcf6f3b000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f16f32c1000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f16f2fbd000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f16f2db9000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f16f2b9f000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f16f292c000)
        libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f16f26c0000)
        libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f16f222d000)
        libxml2.so.2 => /usr/lib/x86_64-linux-gnu/libxml2.so.2 (0x00007f16f1e72000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f16f1c6f000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f16f1a37000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f16f1698000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f16f3a38000)
        libicui18n.so.57 => /usr/lib/x86_64-linux-gnu/libicui18n.so.57 (0x00007f16f121e000)
        libicuuc.so.57 => /usr/lib/x86_64-linux-gnu/libicuuc.so.57 (0x00007f16f0e76000)
        libicudata.so.57 => /usr/lib/x86_64-linux-gnu/libicudata.so.57 (0x00007f16ef3f9000)
        liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f16ef1d3000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f16eee51000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f16eec3a000)

[xrmx]

uwsgi is not crashing, your application (which is not a wsgi application) is raising an assertion because one of its dependencies cannot load a symbol.

[xrmx]

Also looks like a dup of #1030

[azmeuk]

@xrmx I understand it is not strictly uwsgi crashing :)
This issue seem a bit different than #1030 because even a fresh installation of uwsgi with pip install --no-cache does not solve the problem.

[vhongz]

I also have this issue. My python is from pyenv install 2.7.15 && pyenv virtualenv 2.7.15 yp and using uWSGI v2.0.17 (installation command below).

ImportError: dlopen(/Users/vhong/.pyenv/versions/yp/lib/python2.7/site-packages/setproctitle.so, 2): Symbol not found: _Py_GetArgcArgv
  Referenced from: /Users/vhong/.pyenv/versions/yp/lib/python2.7/site-packages/setproctitle.so
  Expected in: flat namespace

Here is the command I used to reinstall/rebuild uwsgi: pip install --upgrade --force-reinstall --no-binary :all: --no-cache-dir uWSGI==2.0.17

It looks like my uwsgi executable has python plugin embedded, but Py_GetArgcArgv is not in the symbol list.

$ nm -gU ~/.pyenv/versions/yp/bin/uwsgi | grep Argv
00000001001792c0 T _PySys_SetArgv
0000000100178f90 T _PySys_SetArgvEx

So, I tried following the quick start guide to install uwsgi without python plugin, and compile the plugin separately (here I downloaded uwsgi from pip):

$ UWSGI_PROFILE=~/temp/pipcache/uwsgi/uwsgi-2.0.17/buildconf/nolang.ini pip install --upgrade --force-reinstall --no-binary :all: ~/temp/pipcache/uwsgi/uwsgi-2.0.17.tar.gz

$ PYTHON=~/.pyenv/versions/yp/bin/python2.7 ~/.pyenv/versions/yp/bin/uwsgi --build-plugin "$HOME/temp/pipcache/uwsgi/uwsgi-2.0.17/plugins/python python27"
[...]
*** python27 plugin built and available in python27_plugin.so ***

But it looks like python27_plugin.so doesn't have Py_GetArgcArgv either:

$ nm -gU python27_plugin.so | grep Argv
00000000000fee50 T _PySys_SetArgv
00000000000feb20 T _PySys_SetArgvEx

To double check, my python binary has the symbol:

$ nm -gU ~/.pyenv/versions/yp/bin/python2.7 | grep Argv
00000001000ee460 T _PySys_SetArgv
00000001000ee130 T _PySys_SetArgvEx
00000001000ff3c0 T _Py_GetArgcArgv

I don't know exactly how python plugin works, but it seems to have skipped the Py_GetArgcArgv symbol for some reason.
It could be because Py_GetArgcArgv is defined in CPython's Modules/main.c https://github.com/python/cpython/blob/master/Modules/main.c#L1804 ?

[vhongz]

On a side note, my coworkers seem to have their uwsgi linked to their Homebrew Python Framework although they also install uwsgi with pip:

$ otool -L $(which uwsgi)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.50.4)
    /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
    /usr/lib/libexpat.1.dylib (compatibility version 7.0.0, current version 8.0.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1452.23.0)
    /usr/local/opt/python@2/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.0)

I'm trying to replicate this setup but the effort hasn't been very fruitful.

[lovemyliwu]

same issue here

[divad]

I have also hit this exact issue, Python 2.7.15 compiled from source, virtualenv, pip installed uwsgi (inside the virtualenv) on Ubuntu 14.04.

[akshay-hazari-lt]

Same issue

[summertian4]

Yes, this perfectly solved my problem.

sudo pip3 install apache-airflow --no-cache

[dingbell]

Yes, this perfectly solved my problem.

sudo pip3 install apache-airflow --no-cache

I still can't solve the problem with this command. Which python do you use?