Jax: pip install problem

Created on 6 Aug 2019 · 12Comments · Source: google/jax

After
pip install --upgrade jax jaxlib # CPU-only version

I get the following error messages:

  ERROR: Complete output from command /home/username/miniconda3/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-13vz5t0k/fastcache/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-2hul3mn8 --python-tag cp37:
  ERROR: running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/fastcache
  copying fastcache/__init__.py -> build/lib.linux-x86_64-3.7/fastcache
  copying fastcache/benchmark.py -> build/lib.linux-x86_64-3.7/fastcache
  creating build/lib.linux-x86_64-3.7/fastcache/tests
  copying fastcache/tests/__init__.py -> build/lib.linux-x86_64-3.7/fastcache/tests
  copying fastcache/tests/test_thread.py -> build/lib.linux-x86_64-3.7/fastcache/tests
  copying fastcache/tests/test_functools.py -> build/lib.linux-x86_64-3.7/fastcache/tests
  copying fastcache/tests/test_clrucache.py -> build/lib.linux-x86_64-3.7/fastcache/tests
  running build_ext
  building 'fastcache._lrucache' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/src
  gcc -pthread -B /home/username/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/username/miniconda3/include/python3.7m -c src/_lrucache.c -o build/temp.linux-x86_64-3.7/src/_lrucache.o
  gcc -pthread -shared -B /home/username/miniconda3/compiler_compat -L/home/username/miniconda3/lib -Wl,-rpath=/home/username/miniconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/src/_lrucache.o -o build/lib.linux-x86_64-3.7/fastcache/_lrucache.cpython-37m-x86_64-linux-gnu.so
  /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
  /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
  /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
  /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
  build/temp.linux-x86_64-3.7/src/_lrucache.o: file not recognized: file format not recognized
  collect2: error: ld returned 1 exit status
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for fastcache
  Running setup.py clean for fastcache
Failed to build fastcache
Installing collected packages: fastcache, jax, jaxlib
  Running setup.py install for fastcache ... error
    ERROR: Complete output from command /home/username/miniconda3/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-13vz5t0k/fastcache/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-nbn7ls65/install-record.txt --single-version-externally-managed --compile:
    ERROR: running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/fastcache
    copying fastcache/__init__.py -> build/lib.linux-x86_64-3.7/fastcache
    copying fastcache/benchmark.py -> build/lib.linux-x86_64-3.7/fastcache
    creating build/lib.linux-x86_64-3.7/fastcache/tests
    copying fastcache/tests/__init__.py -> build/lib.linux-x86_64-3.7/fastcache/tests
    copying fastcache/tests/test_thread.py -> build/lib.linux-x86_64-3.7/fastcache/tests
    copying fastcache/tests/test_functools.py -> build/lib.linux-x86_64-3.7/fastcache/tests
    copying fastcache/tests/test_clrucache.py -> build/lib.linux-x86_64-3.7/fastcache/tests
    running build_ext
    building 'fastcache._lrucache' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/src
    gcc -pthread -B /home/username/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/username/miniconda3/include/python3.7m -c src/_lrucache.c -o build/temp.linux-x86_64-3.7/src/_lrucache.o
    gcc -pthread -shared -B /home/username/miniconda3/compiler_compat -L/home/username/miniconda3/lib -Wl,-rpath=/home/username/miniconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/src/_lrucache.o -o build/lib.linux-x86_64-3.7/fastcache/_lrucache.cpython-37m-x86_64-linux-gnu.so
    /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
    /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
    /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
    /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
    build/temp.linux-x86_64-3.7/src/_lrucache.o: file not recognized: file format not recognized
    collect2: error: ld returned 1 exit status
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command "/home/username/miniconda3/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-13vz5t0k/fastcache/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-nbn7ls65/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-13vz5t0k/fastcache/

This is on a Manjaro Linux system, strangely on an Ubuntu system the pip install works without problems. This might not be caused by JAX, but may be a problem with the system, but not sure.

build contributions welcome

Source

BEpresent

Most helpful comment

@alekk Good news! We've actually already dropped the fastcache dependency at Github head.

Possibly bad news: we did so by dropping Python 2 support (allowing us to make use of Python 3's built in LRU cache.)

I'll make a new jax release so you don't have to install from git.

hawkinsp on 28 Jan 2020

🎉2

All 12 comments

That sounds like a problem with pip install fastcache. Can you try that and see if you see the same problem? If so, I'd file an issue with the authors of that package.

fastcache is one of the JAX dependencies as of recently; if it proves problematic we could look into alternatives.

hawkinsp on 7 Aug 2019

Thanks for the fast response.

This seems correct, I get the same errors:

pip install fastcache
Collecting fastcache
  Using cached https://files.pythonhosted.org/packages/5f/a3/b280cba4b4abfe5f5bdc643e6c9d81bf3b9dc2148a11e5df06b6ba85a560/fastcache-1.1.0.tar.gz
Building wheels for collected packages: fastcache
  Building wheel for fastcache (setup.py) ... error
  ERROR: Complete output from command /home/username/miniconda3/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-u_hgwy7k/fastcache/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-3vj4yzwo --python-tag cp37:
  ERROR: running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.7
  creating build/lib.linux-x86_64-3.7/fastcache
  copying fastcache/__init__.py -> build/lib.linux-x86_64-3.7/fastcache
  copying fastcache/benchmark.py -> build/lib.linux-x86_64-3.7/fastcache
  creating build/lib.linux-x86_64-3.7/fastcache/tests
  copying fastcache/tests/__init__.py -> build/lib.linux-x86_64-3.7/fastcache/tests
  copying fastcache/tests/test_thread.py -> build/lib.linux-x86_64-3.7/fastcache/tests
  copying fastcache/tests/test_functools.py -> build/lib.linux-x86_64-3.7/fastcache/tests
  copying fastcache/tests/test_clrucache.py -> build/lib.linux-x86_64-3.7/fastcache/tests
  running build_ext
  building 'fastcache._lrucache' extension
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/src
  gcc -pthread -B /home/username/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/username/miniconda3/include/python3.7m -c src/_lrucache.c -o build/temp.linux-x86_64-3.7/src/_lrucache.o
  gcc -pthread -shared -B /home/username/miniconda3/compiler_compat -L/home/username/miniconda3/lib -Wl,-rpath=/home/username/miniconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/src/_lrucache.o -o build/lib.linux-x86_64-3.7/fastcache/_lrucache.cpython-37m-x86_64-linux-gnu.so
  /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
  /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
  /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
  /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
  build/temp.linux-x86_64-3.7/src/_lrucache.o: file not recognized: file format not recognized
  collect2: error: ld returned 1 exit status
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for fastcache
  Running setup.py clean for fastcache
Failed to build fastcache
Installing collected packages: fastcache
  Running setup.py install for fastcache ... error
    ERROR: Complete output from command /home/username/miniconda3/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-u_hgwy7k/fastcache/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-pz_ks5js/install-record.txt --single-version-externally-managed --compile:
    ERROR: running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.7
    creating build/lib.linux-x86_64-3.7/fastcache
    copying fastcache/__init__.py -> build/lib.linux-x86_64-3.7/fastcache
    copying fastcache/benchmark.py -> build/lib.linux-x86_64-3.7/fastcache
    creating build/lib.linux-x86_64-3.7/fastcache/tests
    copying fastcache/tests/__init__.py -> build/lib.linux-x86_64-3.7/fastcache/tests
    copying fastcache/tests/test_thread.py -> build/lib.linux-x86_64-3.7/fastcache/tests
    copying fastcache/tests/test_functools.py -> build/lib.linux-x86_64-3.7/fastcache/tests
    copying fastcache/tests/test_clrucache.py -> build/lib.linux-x86_64-3.7/fastcache/tests
    running build_ext
    building 'fastcache._lrucache' extension
    creating build/temp.linux-x86_64-3.7
    creating build/temp.linux-x86_64-3.7/src
    gcc -pthread -B /home/username/miniconda3/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/username/miniconda3/include/python3.7m -c src/_lrucache.c -o build/temp.linux-x86_64-3.7/src/_lrucache.o
    gcc -pthread -shared -B /home/username/miniconda3/compiler_compat -L/home/username/miniconda3/lib -Wl,-rpath=/home/username/miniconda3/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/src/_lrucache.o -o build/lib.linux-x86_64-3.7/fastcache/_lrucache.cpython-37m-x86_64-linux-gnu.so
    /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
    /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
    /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
    /home/username/miniconda3/compiler_compat/ld: build/temp.linux-x86_64-3.7/src/_lrucache.o: unable to initialize decompress status for section .debug_info
    build/temp.linux-x86_64-3.7/src/_lrucache.o: file not recognized: file format not recognized
    collect2: error: ld returned 1 exit status
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command "/home/username/miniconda3/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-u_hgwy7k/fastcache/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-pz_ks5js/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-u_hgwy7k/fastcache/

BEpresent on 7 Aug 2019

Closing this issue since there isn't much we can do on the JAX side to fix it, given it's a fastcache issue.

hawkinsp on 9 Aug 2019

There is a problem installing the latest versions of jaxlib on ubuntu 18.04 (python 3.6.8 running on venv)

I came across this issue while trying to install ngboost

https://github.com/stanfordmlgroup/ngboost/issues/2

abriosi on 13 Oct 2019

Is there anything I can do to help, since it builds correctly when compiled from source?

abriosi on 14 Oct 2019

The main change from jaxlib 0.1.23 to 0.1.24 was to switch from manylinux1 to manylinux2010 as the platform tag on PyPI. Can you try updating your pip version?

jekbradbury on 14 Oct 2019

👍1

@jekbradbury running pip install --upgrade pip effectively solved the issue with jaxlib.

Thank you

abriosi on 15 Oct 2019

I'm on the latest pip and I'm still having this issue. It runs pretty deep: https://github.com/ContinuumIO/anaconda-issues/issues/11152 with links to many other issue threads across github. Conda tries to essentially cross-compile packages against whatever version of glibc/binutils/godknows is in your virtual environment. If you look at the compilation commands you'll see conda provides a compiler_compat prefix which has its own ld. I don't pretend to understand all the subtleties here, but something goes wrong there and one workaround is to move conda's ld out of the way temporarily so that the system's ld is used instead. Another is to conda install gcc binutils so that all compilation and linking is done by conda-provided tools, presumably on mutually compatible versions. But then you might have to reinstall all binary packages in your conda env.

cooijmanstim on 13 Dec 2019

Hi -- I am teaching an optimization course relying on JAX this semester and apparently, students working on Windows are experiencing the fastcache issue when installing JAX with pip under Windows. Is there an easy work around (eg. previous version?). I've told the students to work on colab at the moment, but that is not ideal.

thanks!