Pylint: Fail to install 2.0.0 in Dockerfile based on alpine:latest

[pzelnip]

When installing 2.0.0 in a Docker image based on alpine:latest it fails to build.

Steps to reproduce

1. Use this Dockerfile:

FROM alpine:latest

RUN apk add --no-cache --update python3

RUN python3 -m ensurepip
RUN pip3 install --upgrade pip

RUN pip3 install pylint==2.0.0

1. Run docker build -t pylinttest:latest .
2. Output is:

Sending build context to Docker daemon  23.48MB
Step 1/5 : FROM alpine:latest
 ---> 11cd0b38bc3c
Step 2/5 : RUN apk add --no-cache --update python3
 ---> Running in 919e13976c5e
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
(1/11) Installing libbz2 (1.0.6-r6)
(2/11) Installing expat (2.2.5-r0)
(3/11) Installing libffi (3.2.1-r4)
(4/11) Installing gdbm (1.13-r1)
(5/11) Installing xz-libs (5.2.4-r0)
(6/11) Installing ncurses-terminfo-base (6.1-r0)
(7/11) Installing ncurses-terminfo (6.1-r0)
(8/11) Installing ncurses-libs (6.1-r0)
(9/11) Installing readline (7.0.003-r0)
(10/11) Installing sqlite-libs (3.24.0-r0)
(11/11) Installing python3 (3.6.4-r1)
Executing busybox-1.28.4-r0.trigger
OK: 65 MiB in 24 packages
Removing intermediate container 919e13976c5e
 ---> c9e9764e68e5
Step 3/5 : RUN python3 -m ensurepip
 ---> Running in f004c83d2550
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages
Requirement already satisfied: pip in /usr/lib/python3.6/site-packages
Removing intermediate container f004c83d2550
 ---> 03392c659e88
Step 4/5 : RUN pip3 install --upgrade pip
 ---> Running in 70465b64883c
Collecting pip
  Downloading https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl (1.3MB)
Installing collected packages: pip
  Found existing installation: pip 9.0.1
    Uninstalling pip-9.0.1:
      Successfully uninstalled pip-9.0.1
Successfully installed pip-10.0.1
Removing intermediate container 70465b64883c
 ---> aecf57ec2910
Step 5/5 : RUN pip3 install pylint==2.0.0
 ---> Running in 3220b0a0f750
Collecting pylint==2.0.0
  Downloading https://files.pythonhosted.org/packages/87/ef/7d20e99471e4779583812be3d768818c0fca89ffad6f19ffbc1f167ebe07/pylint-2.0.0-py3-none-any.whl (732kB)
Collecting isort>=4.2.5 (from pylint==2.0.0)
  Downloading https://files.pythonhosted.org/packages/1f/2c/22eee714d7199ae0464beda6ad5fedec8fee6a2f7ffd1e8f1840928fe318/isort-4.3.4-py3-none-any.whl (45kB)
Collecting mccabe (from pylint==2.0.0)
  Downloading https://files.pythonhosted.org/packages/87/89/479dc97e18549e21354893e4ee4ef36db1d237534982482c3681ee6e7b57/mccabe-0.6.1-py2.py3-none-any.whl
Collecting astroid>=2.0.0 (from pylint==2.0.0)
  Downloading https://files.pythonhosted.org/packages/2f/82/639f0508f6211a799326f28d551a940c632fa6f766541878225e476df270/astroid-2.0-py3-none-any.whl (163kB)
Collecting wrapt (from astroid>=2.0.0->pylint==2.0.0)
  Downloading https://files.pythonhosted.org/packages/a0/47/66897906448185fcb77fc3c2b1bc20ed0ecca81a0f2f88eda3fc5a34fc3d/wrapt-1.10.11.tar.gz
Collecting typed-ast; python_version < "3.7" and implementation_name == "cpython" (from astroid>=2.0.0->pylint==2.0.0)
  Downloading https://files.pythonhosted.org/packages/52/cf/2ebc7d282f026e21eed4987e42e10964a077c13cfc168b42f3573a7f178c/typed-ast-1.1.0.tar.gz (200kB)
Collecting six (from astroid>=2.0.0->pylint==2.0.0)
  Downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting lazy-object-proxy (from astroid>=2.0.0->pylint==2.0.0)
  Downloading https://files.pythonhosted.org/packages/55/08/23c0753599bdec1aec273e322f277c4e875150325f565017f6280549f554/lazy-object-proxy-1.3.1.tar.gz
Installing collected packages: isort, mccabe, wrapt, typed-ast, six, lazy-object-proxy, astroid, pylint
  Running setup.py install for wrapt: started
    Running setup.py install for wrapt: finished with status 'done'
  Running setup.py install for typed-ast: started
    Running setup.py install for typed-ast: finished with status 'error'
    Complete output from command /usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-ep32w8yg/typed-ast/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-i8sgnfk4/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/typed_ast
    copying typed_ast/ast27.py -> build/lib.linux-x86_64-3.6/typed_ast
    copying typed_ast/__init__.py -> build/lib.linux-x86_64-3.6/typed_ast
    copying typed_ast/conversions.py -> build/lib.linux-x86_64-3.6/typed_ast
    copying typed_ast/ast3.py -> build/lib.linux-x86_64-3.6/typed_ast
    running build_ext
    building '_ast27' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/ast27
    creating build/temp.linux-x86_64-3.6/ast27/Parser
    creating build/temp.linux-x86_64-3.6/ast27/Python
    creating build/temp.linux-x86_64-3.6/ast27/Custom
    gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Os -fomit-frame-pointer -g -Os -fomit-frame-pointer -g -Os -fomit-frame-pointer -g -DTHREAD_STACK_SIZE=0x100000 -fPIC -Iast27/Include -I/usr/include/python3.6m -c ast27/Parser/acceler.c -o build/temp.linux-x86_64-3.6/ast27/Parser/acceler.o
    unable to execute 'gcc': No such file or directory
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-ep32w8yg/typed-ast/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-i8sgnfk4/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-ep32w8yg/typed-ast/
The command '/bin/sh -c pip3 install pylint==2.0.0' returned a non-zero code: 1

Current behavior

With 2.0.0 it fails, but with 1.9.2 the same build steps complete successfully

Expected behavior

That the pip install should complete successfully.

pylint --version output

Trying to install 2.0.0, but failing. 1.9.2 completes successfully.

[brycepg]

astroid has a new dependency typed-ast for CPython < 3.7. It does have a wheel under the manylinux1 tag, who depends on some shared objects that are not available in alpine linux I think (see https://www.python.org/dev/peps/pep-0513/#the-manylinux1-policy).

Possible workarounds:

• install gcc and such to make the compilation work
• use python 3.7

Might be a good issue for typed-ast

Also see https://github.com/pypa/pip/issues/3969

[brycepg]

Here's an alpine Dockerfile which works around this issue by telling pip that the typed_ast manylinux wheel is compatible with this system. It appears that their is no ABI issue:

FROM alpine:latest

RUN apk add --no-cache --update python3
RUN python3 -m ensurepip
RUN pip3 install --upgrade pip
RUN python3 -c 'import sys; f = open("/usr/lib/python3.6/site-packages/_manylinux.py", "w"); f.write("manylinux1_compatible = True"); f.close()'
RUN python3 -m pip install --upgrade astroid

[pzelnip]

For the record in case anyone else ends up here, to get it working I added:

RUN apk add --no-cache --update python3-dev  gcc build-base

to my Dockerfile and now pylint 2.0.0 installs successfully.

[aleksandr-kotlyar]

Problem seems to be returned in 2.4.4 pylint version with python 3.7.

update1
@pzelnip Thank you. Your solution with gcc works, but too heavy, about +100 MB, for example, for me it's (x3) larger container. I will try to find another.

update2 (after 8 hours)
@brycepg Thank you so much! Your solution is lightweight and working!
I shorted it a bit and fixed path to _manylinux.py in open("..")
Finally, after eight hours, here is my final Dockerfile where requirements.txt contains pylint==2.4.4

FROM python:3.7-alpine
COPY requirements.txt /app/requirements.txt
WORKDIR /app
RUN python3 -c 'import sys; f = open("/usr/local/lib/python3.7/site-packages/_manylinux.py", "w"); f.write("manylinux1_compatible = True"); f.close()'
RUN pip install --no-cache-dir -r requirements.txt

[twolaw]

You can use @pzelnip solution and keep it lightweight by doing this :

RUN apk add --no-cache --virtual .build-deps python3-dev gcc build-base \
 && pip install --no-cache-dir -r requirements.txt \
 && apk del .build-deps

A virtual package is created containing build stuff, and it is removed as soon as pip install finished.
Because apk add and apk del are in the same RUN instruction, it does not add weight to your image.

[Xerrex]

Hello, Does this still work while using a virtual environment for your application.?