black 19.10b0 versoin of pip install fails in python alpine docker image

[yadachi]

Hi new version of black 19.10b0 giving error when installing in python:3.7.5-alpine3.10 docker image. (python:3.7.5-buster/stretch are successfully installed)
only happening in python alpine image.
typed-ast seems to be not compatible in python alpine image.

Building wheels for collected packages: typed-ast, regex, pathspec Building wheel for typed-ast (setup.py): started Building wheel for typed-ast (setup.py): finished with status 'error' ERROR: Command errored out with exit status 1: command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ireshofk/typed-ast/setup.py'"'"'; __file__='"'"'/tmp/pip-install-ireshofk/typed-ast/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ub66nw4b --python-tag cp37 cwd: /tmp/pip-install-ireshofk/typed-ast/

[williamjacksn]

You need to install the gcc and musl-dev packages before installing black. Here is an example Dockerfile:

FROM python:3.7.5-alpine3.10

RUN /sbin/apk add --no-cache --virtual .deps gcc musl-dev \
 && /usr/local/bin/pip install --no-cache-dir black==19.10b0 \
 && /sbin/apk del --no-cache .deps

[ambv]

We need typed-ast. Do you have an idea how this is actionable for us?

[uhinze]

My full log output in case it helps for troubleshooting:

Collecting black
  Downloading https://files.pythonhosted.org/packages/fd/bb/ad34bbc93d1bea3de086d7c59e528d4a503ac8fe318bd1fa48605584c3d2/black-19.10b0-py36-none-any.whl (97kB)

     |████████████████████████████████| 102kB 25.0MB/s 
Collecting toml>=0.9.4
  Downloading https://files.pythonhosted.org/packages/a2/12/ced7105d2de62fa7c8fb5fce92cc4ce66b57c95fb875e9318dba7f8c5db0/toml-0.10.0-py2.py3-none-any.whl
Collecting regex
  Downloading https://files.pythonhosted.org/packages/6f/a6/99eeb5904ab763db87af4bd71d9b1dfdd9792681240657a4c0a599c10a81/regex-2019.08.19.tar.gz (654kB)

     |████████████████████████████████| 655kB 71.1MB/s 
Collecting pathspec<1,>=0.6
  Downloading https://files.pythonhosted.org/packages/7a/68/5902e8cd7f7b17c5879982a3a3ee2ad0c3b92b80c79989a2d3e1ca8d29e1/pathspec-0.6.0.tar.gz
Collecting click>=6.5
  Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)

     |████████████████████████████████| 81kB 17.9MB/s 
Collecting appdirs
  Downloading https://files.pythonhosted.org/packages/56/eb/810e700ed1349edde4cbdc1b2a21e28cdf115f9faf263f6bbf8447c1abf3/appdirs-1.4.3-py2.py3-none-any.whl
Collecting typed-ast>=1.4.0
  Downloading https://files.pythonhosted.org/packages/34/de/d0cfe2ea7ddfd8b2b8374ed2e04eeb08b6ee6e1e84081d151341bba596e5/typed_ast-1.4.0.tar.gz (206kB)

     |████████████████████████████████| 215kB 75.6MB/s 
Collecting attrs>=18.1.0
  Downloading https://files.pythonhosted.org/packages/a2/db/4313ab3be961f7a763066401fb77f7748373b6094076ae2bda2806988af6/attrs-19.3.0-py2.py3-none-any.whl
Building wheels for collected packages: regex, pathspec, typed-ast
  Building wheel for regex (setup.py) ... - error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-qnjpob9a/regex/setup.py'"'"'; __file__='"'"'/tmp/pip-install-qnjpob9a/regex/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-sydltezx --python-tag cp37
       cwd: /tmp/pip-install-qnjpob9a/regex/
  Complete output (22 lines):
  BASE_DIR is /tmp/pip-install-qnjpob9a/regex
  /usr/local/lib/python3.7/site-packages/setuptools/dist.py:475: UserWarning: Normalizing '2019.08.19' to '2019.8.19'
    normalized_version,
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/regex
  copying regex_3/regex/__init__.py -> build/lib.linux-x86_64-3.7/regex
  copying regex_3/regex/regex.py -> build/lib.linux-x86_64-3.7/regex
  copying regex_3/regex/_regex_core.py -> build/lib.linux-x86_64-3.7/regex
  creating build/lib.linux-x86_64-3.7/regex/test
  copying regex_3/regex/test/__init__.py -> build/lib.linux-x86_64-3.7/regex/test
  copying regex_3/regex/test/test_regex.py -> build/lib.linux-x86_64-3.7/regex/test
  running build_ext
  building 'regex._regex' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/regex_3
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/local/include/python3.7m -c regex_3/_regex.c -o build/temp.linux-x86_64-3.7/regex_3/_regex.o
  unable to execute 'gcc': No such file or directory
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for regex
  Running setup.py clean for regex
  Building wheel for pathspec (setup.py) ... - \ done
  Created wheel for pathspec: filename=pathspec-0.6.0-cp37-none-any.whl size=26671 sha256=f82e6322289a5c22aa7ed4f375cd3036ff3fa647a0bf7e4cc7f37dc7d99f952d
  Stored in directory: /root/.cache/pip/wheels/62/b8/e1/e2719465b5947c40cd85d613d6cb33449b86a1ca5a6c574269
  Building wheel for typed-ast (setup.py) ... - error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-qnjpob9a/typed-ast/setup.py'"'"'; __file__='"'"'/tmp/pip-install-qnjpob9a/typed-ast/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-3z2e795w --python-tag cp37
       cwd: /tmp/pip-install-qnjpob9a/typed-ast/
  Complete output (23 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/typed_ast
  copying typed_ast/__init__.py -> build/lib.linux-x86_64-3.7/typed_ast
  copying typed_ast/conversions.py -> build/lib.linux-x86_64-3.7/typed_ast
  copying typed_ast/ast27.py -> build/lib.linux-x86_64-3.7/typed_ast
  copying typed_ast/ast3.py -> build/lib.linux-x86_64-3.7/typed_ast
  package init file 'ast3/tests/__init__.py' not found (or not a regular file)
  creating build/lib.linux-x86_64-3.7/typed_ast/tests
  copying ast3/tests/test_basics.py -> build/lib.linux-x86_64-3.7/typed_ast/tests
  running build_ext
  building '_ast27' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/ast27
  creating build/temp.linux-x86_64-3.7/ast27/Parser
  creating build/temp.linux-x86_64-3.7/ast27/Python
  creating build/temp.linux-x86_64-3.7/ast27/Custom
  gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -Iast27/Include -I/usr/local/include/python3.7m -c ast27/Parser/acceler.c -o build/temp.linux-x86_64-3.7/ast27/Parser/acceler.o
  unable to execute 'gcc': No such file or directory
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for typed-ast
  Running setup.py clean for typed-ast
Successfully built pathspec
Failed to build regex typed-ast
Installing collected packages: toml, regex, pathspec, click, appdirs, typed-ast, attrs, black
    Running setup.py install for regex ... - error
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-qnjpob9a/regex/setup.py'"'"'; __file__='"'"'/tmp/pip-install-qnjpob9a/regex/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-1_x31o3r/install-record.txt --single-version-externally-managed --compile
         cwd: /tmp/pip-install-qnjpob9a/regex/
    Complete output (22 lines):
    BASE_DIR is /tmp/pip-install-qnjpob9a/regex
    /usr/local/lib/python3.7/site-packages/setuptools/dist.py:475: UserWarning: Normalizing '2019.08.19' to '2019.8.19'
      normalized_version,
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/regex
    copying regex_3/regex/__init__.py -> build/lib.linux-x86_64-3.7/regex
    copying regex_3/regex/regex.py -> build/lib.linux-x86_64-3.7/regex
    copying regex_3/regex/_regex_core.py -> build/lib.linux-x86_64-3.7/regex
    creating build/lib.linux-x86_64-3.7/regex/test
    copying regex_3/regex/test/__init__.py -> build/lib.linux-x86_64-3.7/regex/test
    copying regex_3/regex/test/test_regex.py -> build/lib.linux-x86_64-3.7/regex/test
    running build_ext
    building 'regex._regex' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/regex_3
    gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/local/include/python3.7m -c regex_3/_regex.c -o build/temp.linux-x86_64-3.7/regex_3/_regex.o
    unable to execute 'gcc': No such file or directory
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-qnjpob9a/regex/setup.py'"'"'; __file__='"'"'/tmp/pip-install-qnjpob9a/regex/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-1_x31o3r/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.

[uhinze]

I'm pinning to the previous version in my CI for now:
pip install black==19.3b0

[ambv]

@uhinze, have you checked @williamjacksn's suggestion?

[uhinze]

@ambv nope just did the pinning as that's all I need for now

[ambv]

@uhinze The error says you're missing gcc in your image. Installing that will allow you to use newer Black.

[joekohlsdorf]

This happens on all platforms which don't have a compiler installed as the regex package doesn't have wheels.

Introduced in https://github.com/psf/black/pull/1047

[yadachi]

@williamjacksn
installing gcc before black installation works!
Thank you.

[jezdez]

@ambv It's not so great that there is a C compiler dependency now for black, is there a different way to solve #455 than that?

[jezdez]

The regex project author asks contributors to provide Linux wheels in https://bitbucket.org/mrabarnett/mrab-regex/issues/343/wheel-for-linux because they are only able to build them manually on Windows 10.

[hugovk]

This is being discussed in the duplicate issue: https://github.com/psf/black/issues/1112#issuecomment-548751139.

[jezdez]

Thanks @hugovk!

[dibyo09]

The code from williamjacksn commented on Oct 29, 2019 • helped me to resolved the issue

RUN /sbin/apk add --no-cache --virtual .deps gcc musl-dev \
&& /usr/local/bin/pip install --no-cache-dir black==19.10b0 \
&& /sbin/apk del --no-cache .deps

Thanks

You need to install the gcc and musl-dev packages before installing black. Here is an example Dockerfile:

FROM python:3.7.5-alpine3.10

RUN /sbin/apk add --no-cache --virtual .deps gcc musl-dev \
 && /usr/local/bin/pip install --no-cache-dir black==19.10b0 \
 && /sbin/apk del --no-cache .deps