Nuitka: Portable binary with musl

[anki-code]

Hello! Thank you for Nuitka!

Could you please advice what is the best way to make standalone binary with portable musl wich will work portable on Ubuntu and Alpine? I tried and looks like close to success.

Here is the example and additional questions below:

1. I build the xxh project (it hasn't any complex libs) with Nuitka:

docker run --rm -it -v `pwd`:/host alpine sh
apk add --update musl-dev gcc python3-dev py3-pip chrpath git vim mc
pip3 install -U "https://github.com/Nuitka/Nuitka/archive/factory.zip"
pip3 install pexpect pyyaml
git clone --depth 1 https://github.com/xxh/xxh
cd xxh
nuitka3 --python-flag=no_site --python-flag=no_warnings \
        --show-progress --standalone --follow-imports xxh

1. After it I have xxh.dist with ldd xxh on Alpine:

/lib/ld-musl-x86_64.so.1 (0x7f58f86fb000)
libpython3.8.so.1.0 => ./libpython3.8.so.1.0 (0x7f58f7a96000)
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f58f86fb000)

The app works perfect on Alpine.

1. Then I move it to Ubuntu and I've got ldd xxh:

linux-vdso.so.1 (0x00007ffeeedd9000)
libpython3.8.so.1.0 => /home/pc/xxh/xxh/xxh.dist/./libpython3.8.so.1.0 (0x00007efd2813c000)
libc.musl-x86_64.so.1 => not found
libc.musl-x86_64.so.1 => not found

1. When I copy libc.musl-x86_64.so.1 from Alpine to xxh.dist
   I've got ldd xxh on Ubuntu:

linux-vdso.so.1 (0x00007ffce37ac000)
libpython3.8.so.1.0 => /home/pc/xxh/xxh/xxh.dist/./libpython3.8.so.1.0 (0x00007fbbf0df5000)
libc.musl-x86_64.so.1 => /home/pc/xxh/xxh/xxh.dist/./libc.musl-x86_64.so.1 (0x00007fbbf0d5f000)

1. Looks good, but not working:

ubuntu$ ./xxh
bash: ./xxh: No such file or directory

1. And only if I copy musl to /lib on Ubuntu I have success:

$ sudo cp libc.musl-x86_64.so.1 /lib/ld-musl-x86_64.so.1
$ ./xxh -V
0.6.0

1. If I open xxh binary in vim I see absolute path /lib/ld-musl-x86_64.so.1 and it looks like the cause why I should copy the musl to system lib directory.

A. Please advice what I can do to make musl portable with the app?
B. Is there a way to statically compile musl inside the app?
C. Is there a way to make one big static binary xxh file?

Musl is small and some projects use it instead of libc for creating static standalone apps. It will be great if Nuitka can do the same.

Many thanks to your work!

[kayhayen]

The glibc cannot be included, simply because it fails to work with older kernels. We therefore have code that specifically inhibits it.

I am not versed with muslc at all. Does it not have the same issue. Are you saying that there is a portable version of muslc and there is a trick to use it? Nuitka surely could change how to links and statically link that. Just let me know which flags you need to add for that to gcc and ld. You can export CCFLAGS= and LDFLAGS= and Nuitka will include it for your experimentation.

[anki-code]

Nuitka surely could change how to links and statically link that. Just let me know which flags you need to add for that to gcc and ld. You can export CCFLAGS= and LDFLAGS= and Nuitka will include it for your experimentation.

Hello @romkatv! Could your experience of building zsh-bin with static musl be applied here?

[romkatv]

You need to enable static linking. If you are using gcc, try LDFLAGS=-static.

[anki-code]

Next steps. I've built static python as described in wiki and got libpython3.8.a.

wget https://www.python.org/ftp/python/3.8.2/Python-3.8.2.tgz
cd Python3.8.2
./configure LDFLAGS="-static" --disable-shared
make LDFLAGS="-static" LINKFORSHARED=" "
cp libpython3.8.a /usr/lib

Next I tried to static build with LDFLAGS:

export LDFLAGS="-static -l:libpython3.8.a"
nuitka3 --python-flag=no_site --python-flag=no_warnings --show-progress \
        --standalone --follow-imports xxh

After this I've got xxh.dist with ldd xxh:

/lib/ld-musl-x86_64.so.1 (0x7f890d65e000)

And on Ubuntu ldd shows:

statically linked

It's much better than before!

But when I tried to run ./xxh I've got:

Traceback (most recent call last):
  File "/xxh/xxh.dist/xxh", line 5, in <module>
  File "/xxh/xxh.dist/xxh_xxh/xxh.py", line 1, in <module xxh_xxh.xxh>
  File "/xxh/xxh.dist/yaml/__init__.py", line 8, in <module yaml>
  File "/xxh/xxh.dist/yaml/loader.py", line 8, in <module yaml.loader>
  File "/xxh/xxh.dist/yaml/constructor.py", line 14, in <module yaml.constructor>
  File "/xxh/xxh.dist/datetime.py", line 8, in <module>
ImportError: Dynamic loading not supported

[anki-code]

I've got success!

  ...
  File "/xxh/xxh.dist/yaml/constructor.py", line 14, in <module yaml.constructor>
  File "/xxh/xxh.dist/datetime.py", line 8, in <module>
ImportError: Dynamic loading not supported

This error means that on line 8 in Python standard datetime.py file there is an import of module which not included to Python static while building.

To fix this:

1. Find datetime.py (/usr/lib/python3.8/datetime.py on Alpine) and get the module name from line 8 (it's math module)
2. Open Python3.8.2/Modules/Setup, search the module and uncomment:

math mathmodule.c _math.c

1. Rebuild static Python and copy to /usr/lib:

./configure LDFLAGS="-static" --disable-shared
make LDFLAGS="-static" LINKFORSHARED=" "
cp libpython3.8.a /usr/lib

1. Now try to rebuild app with Nuitka:

export LDFLAGS="-static -l:libpython3.8.a"
nuitka3 --python-flag=no_site --python-flag=no_warnings --show-progress \
        --standalone --follow-imports xxh

1. If you faced with error go to step 1 and repeat until getting all required modules to static Python
2. If you have success you've got the static linked binary of your app which portable on Alpine, Ubuntu, Arch, CentOS (I've tested):

alpine# ldd xxh.dist/xxh
/lib/ld-musl-x86_64.so.1 (0x7f890d65e000)
ubuntu# ldd xxh.dist/xxh
statically linked