Python: Cannot install torch 0.4.1.post2 on python:3.7-alpine

[JesseTG]

I'm trying to install PyTorch version 0.4.1 on the python:3.7-alpine image. However, pip3.7 only sees 0.1.2:

/ # pip3.7 install --no-cache-dir torch
Collecting torch
  Downloading https://files.pythonhosted.org/packages/5f/e9/bac4204fe9cb1a002ec6140b47f51affda1655379fe302a1caef421f9846/torch-0.1.2.post1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-xg7fjq4l/torch/setup.py", line 11, in <module>
        raise RuntimeError(README)
    RuntimeError: PyTorch does not currently provide packages for PyPI (see status at https://github.com/pytorch/pytorch/issues/566).

    Please follow the instructions at http://pytorch.org/ to install with miniconda instead.


    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-xg7fjq4l/torch/
/ # 

And yet, when searching for torch...

/ # pip3.7 search torch
<snip for brevity>
torch (0.4.1.post2)        - Tensors and Dynamic neural networks in Python with strong GPU acceleration
<snip for brevity>
/ # 

...you see that pip3.7 finds version 0.4.1.post2.

[wglambert]

I'm not familiar with python; it seems for the latest version the download is offered in a .whl format which "is not a supported wheel on this platform". However 0.1.2.post1 is offered in .tar.gz so I'm assuming pip is detecting the platform and only allowing compatible installs.
https://packaging.python.org/tutorials/installing-packages/#source-distributions-vs-wheels

On the debian variant the package installs correctly, I might also add that the package is quite large relative to the base image at 497MB compressed; eclipsing the benefit of using an alpine base for the purpose of a smaller size.

$ ls -lh
total 620M
-rw-rw-r-- 1 rei rei  28M Oct  1 11:26 python:3.7-alpine.tar.gz
-rw-rw-r-- 1 rei rei  48M Oct  1 11:23 python:3.7-slim-stretch.tar.gz
-rw-rw-r-- 1 rei rei 545M Oct  1 11:27 pytorch-python:3.7-slim-stretch.tar.gz

[JesseTG]

I don't understand much about how Python software is actually packaged and distributed, but why wouldn't a wheel be supported on Alpine?

[wglambert]

Alpine uses a different libc: musl, compared to glibc. And the .whl format is not compatible with musl. And also that some packages for pip are only offered in .whl format
https://github.com/docker-library/docs/issues/904
https://github.com/pypa/manylinux/issues/37
https://github.com/pypa/pip/issues/3969

https://github.com/pypa/manylinux

The manylinux1 tags allow projects to distribute wheels that are automatically installed (and work!) on the vast majority of desktop and server Linux distributions.

Docs on the alpine image variant
https://github.com/docker-library/docs/tree/master/python#pythonversion-alpine

[JayH5]

To be clear, _manylinux1_ wheels with native extensions can't be installed on Alpine Linux. Wheels that are pure Python code (_none_ wheels) work just fine on Alpine.

If you add the --verbose flag to your pip command you can see it rejecting the other wheels and versions.

[JesseTG]

I see. So you can't do anything about this issue without doing one of the following:

1. Adding support for a slim distribution that uses glibc instead of musl
2. Adding musl support to pytorch
3. Making a glibc-based alpine

Is my assessment correct?

[JayH5]

1. Adding support for a slim distribution that uses glibc instead of musl

The python:3.7-slim-stretch image that @wglambert suggested is "slim" and uses glibc. I would also recommend that you use an image like that.

1. Adding musl support to pytorch

It's not that PyTorch definitely doesn't support musl, it just doesn't build packages for it. There also isn't a Python wheel format that can be uploaded to PyPI that supports musl. You _could probably_ compile PyTorch from source for Alpine if you really wanted to but it probably wouldn't be worth the effort.

1. Making a glibc-based alpine

While such a thing _does_ exist, I wouldn't recommend it as it can cause all kinds of problems. One of the major things in Alpine that makes it so small is that it uses musl _and not_ glibc. You may as well just use one of the Debian-based slim images.

[tianon]

(Closing, since there's nothing more we can do here.)

[JesseTG]

Thank you anyway.