Pyenv: ImportError: No module named pyexpat

[ymkjp]

I got error on OS X 10.11.3 when I try to install v2.7.10 as following:

$ xcode-select --install
$ brew update && brew install pyenv-virtualenv
$ eval "$(pyenv init -)"
$ eval "$(pyenv virtualenv-init -)"
$ CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install -v 2.7.10

error: failed to install pip via get-pip.py

BUILD FAILED (OS X 10.11.3 using python-build 20151028)

Inspect or clean up the working tree at /var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850
Results logged to /var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850.log

Last 10 log lines:
    info = _mac_ver_xml()
  File "/Users/kenta.ky.yamamoto/.pyenv/versions/2.7.10/lib/python2.7/platform.py", line 786, in _mac_ver_xml
    pl = plistlib.readPlist(fn)
  File "/Users/kenta.ky.yamamoto/.pyenv/versions/2.7.10/lib/python2.7/plistlib.py", line 78, in readPlist
    rootObject = p.parse(pathOrFile)
  File "/Users/kenta.ky.yamamoto/.pyenv/versions/2.7.10/lib/python2.7/plistlib.py", line 401, in parse
    from xml.parsers.expat import ParserCreate
  File "/Users/kenta.ky.yamamoto/.pyenv/versions/2.7.10/lib/python2.7/xml/parsers/expat.py", line 4, in <module>
    from pyexpat import *
ImportError: No module named pyexpat

python-build.20160215184029.70850.log

Here's whole output in /var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850.log.

I recently update my OS X to Mavericks and it may cause this error, but I have no idea to solve this.
Any suggestions?

Thanks.

[yyuu]

Your system's expat installation is broken, or there's broken build in /usr/local/lib or so. Don't know why, though. It's a problem of yours anyways.

*** WARNING: renaming "pyexpat" since importing it failed: dlopen(build/lib.macosx-10.11-x86_64-2.7/pyexpat.so, 2): Symbol not found: _XML_ErrorString
  Referenced from: build/lib.macosx-10.11-x86_64-2.7/pyexpat.so
  Expected in: flat namespace
 in build/lib.macosx-10.11-x86_64-2.7/pyexpat.so
building '_elementtree' extension
clang -fno-strict-aliasing -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DHAVE_EXPAT_CONFIG_H=1 -DUSE_PYEXPAT_CAPI -I/private/var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850/Python-2.7.10/./Modules/expat -I/private/var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850/Python-2.7.10/Mac/Include -I/private/var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850/Python-2.7.10/Mac/Include -I. -IInclude -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/usr/local/include -I/private/var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850/Python-2.7.10/Include -I/private/var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850/Python-2.7.10 -c /private/var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850/Python-2.7.10/Modules/_elementtree.c -o build/temp.macosx-10.11-x86_64-2.7/private/var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850/Python-2.7.10/Modules/_elementtree.o
clang -bundle -undefined dynamic_lookup -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/Users/kenta.ky.yamamoto/.pyenv/versions/2.7.10/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/Users/kenta.ky.yamamoto/.pyenv/versions/2.7.10/lib -fno-strict-aliasing -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/usr/local/opt/openssl/include -I/Users/kenta.ky.yamamoto/.pyenv/versions/2.7.10/include build/temp.macosx-10.11-x86_64-2.7/private/var/folders/mt/vnl_kg3s72dgych6j2j4djxmlbfpd_/T/python-build.20160215184029.70850/Python-2.7.10/Modules/_elementtree.o -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/Users/kenta.ky.yamamoto/.pyenv/versions/2.7.10/lib -L/usr/local/lib -o build/lib.macosx-10.11-x86_64-2.7/_elementtree.so

[9nix00]

I have same issue on Mac and I solved it. the problem is you use an old MacOS X but use a newer Xcode. you have to upgrade your system to Sierra.

[murawskikrzysiek]

I am using Sierra and also seeing this issue...

[9nix00]

@murawskikrzysiek remove your C_INCLUDE_PATH LDFLAGS variable ...etc... and try it again.

[justinjdickow]

Also having this issue running pyenv install 2.7.14 on MacOS 10.13.2

[9nix00]

@justinjdickow last week, this error caused again for me...

you can try this

• make sure your pyenv is latest version and install from brew.
• PKG_CONFIG_PATH="$(brew --prefix openssl)/lib/pkgconfig" pyenv install YOUR-VERSION

[justinjdickow]

@wangwenpei I'm still seeing the same issue

my pyenv is version 1.2.0

I'm using the following flags now

export CFLAGS="-I$(xcrun --show-sdk-path)/usr/include"
export CFLAGS="-I$(brew --prefix readline)/include $CFLAGS"
export LDFLAGS="-L$(brew --prefix readline)/lib $LDFLAGS"
export CFLAGS="-I$(brew --prefix openssl)/include $CFLAGS"
export LDFLAGS="-L$(brew --prefix openssl)/lib $LDFLAGS"
export PKG_CONFIG_PATH="$(brew --prefix openssl)/lib/pkgconfig"

[jalevin]

Besides pyenv I also could not install ruby versions via rbenv. At this point I started looking at the system. I'd recently upgraded and used the migration assistant to move to a new machine.

1. go to Applications and run Xcode. I had additional packages install
2. run brew doctor. if any files in /usr/local/include show up you have 2 options. Either rename the directory to something else like include_old or remove the offending entries. I chose the first.

sudo mv /usr/local/include /usr/local/include_old
sudo mkdir /usr/local/include
sudo chown $YOU_USERNAME:admin /usr/local/include

afterward, I was able to install ruby and python versions

[henriquebastos]

This worked for me:

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk MACOSX_DEPLOYMENT_TARGET=10.14 pyenv install 3.7.3

[danodonovan]

@henriquebastos solution also worked for me, installing 3.7.4

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk \
MACOSX_DEPLOYMENT_TARGET=10.14 \
pyenv install 3.7.4

[pancodia]

Solution from @henriquebastos also work for me

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk \
> MACOSX_DEPLOYMENT_TARGET=10.14 \
> python-build 3.6.8 ~/.runtimes/Python36

[a-ninja]

Solution from @henriquebastos also work for me:
SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk \

MACOSX_DEPLOYMENT_TARGET=10.14 \
pyenv install 3.7.4

[waffletower]

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk MACOSX_DEPLOYMENT_TARGET=10.14 pyenv install 3.7.

Would be really nice if these envvar dependencies were instrumented in pyenv install with sensible defaults on MacOS. For example, the above path is really just a symbolic link to: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk which remains stable release to release.

I also had success building 3.7.4 using:

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk \
pyenv install 3.7.4

without setting an explicit deployment target.

[fnl]

Had the same problem when building (any) python version(s), ModuleNotFoundError: No module named 'pyexpat' and fixed it with the $SDKROOT setup described in this (closed?) ticket. Thank you, @henriquebastos !

[palmerc]

This comes up again. From the log you can see that 'pyenv install 3.7.4' fails to link expat because when linking pyexpat it doesn't try to link in expat and therefore the symbol XML_ErrorString is missing.

clang -bundle -undefined dynamic_lookup -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/Users/admin/.pyenv/versions/3.7.4/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/Users/admin/.pyenv/versions/3.7.4/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/Users/admin/.pyenv/versions/3.7.4/lib -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/Users/admin/.pyenv/versions/3.7.4/include build/temp.macosx-10.14-x86_64-3.7/private/var/folders/x0/9tltnhy90v7fptftj3b7bl9r0000gn/T/python-build.20190913105722.52617/Python-3.7.4/Modules/pyexpat.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/x0/9tltnhy90v7fptftj3b7bl9r0000gn/T/python-build.20190913105722.52617/Python-3.7.4/Modules/expat/xmlparse.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/x0/9tltnhy90v7fptftj3b7bl9r0000gn/T/python-build.20190913105722.52617/Python-3.7.4/Modules/expat/xmlrole.o build/temp.macosx-10.14-x86_64-3.7/private/var/folders/x0/9tltnhy90v7fptftj3b7bl9r0000gn/T/python-build.20190913105722.52617/Python-3.7.4/Modules/expat/xmltok.o -L/usr/local/opt/readline/lib -L/Users/admin/.pyenv/versions/3.7.4/lib -L/usr/local/lib -o build/lib.macosx-10.14-x86_64-3.7/pyexpat.cpython-37m-darwin.so

[palmerc]

So there pyexpatns.h renames a bunch of symbols and that seems to be the root of the trouble

[palmerc]

Building using the same tarball as pyenv downloads, configure and make works.

So there is definitely an include path problem when building with pyenv.

This patch when applied to Python using pyenv fixes the install. Essentially I rename expat.h to expat_internal.h and expat_external.h to expat_external_internal.h.

[robgolding]

Note that if you have the Xcode Command Line Tools installed, but not Xcode.app itself, you can use /Library/Developer/CommandLineTools/SDKs/ instead:

SDKROOT=/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk MACOSX_DEPLOYMENT_TARGET=10.14 pyenv install ...

[Zeesy]

@robgolding 's suggestion worked great for me after having some difficulty with @henriquebastos 's. Running version install for 3.5.3 on MacOS 10.14.

I'd note for anyone having difficulty that install for v 3.5.2 fails regardless of install method on both Ubuntu and MacOS.

[terrisgit]

See also https://github.com/pyenv/pyenv/issues/950#issuecomment-662525606