Pyenv: Install failed, "zlib not available" on macOS Mojave

[tshu-w]

Description

• [x] Platform information (e.g. Ubuntu Linux 16.04): macOS Mojave 10.14
• [x] OS architecture (e.g. amd64):amd64
• [x] pyenv version: 1.2.7
• [x] Python version: 3.7.0 & 2.7.15
• [x] C Compiler information (e.g. gcc 7.3): Apple LLVM version 10.0.0 (clang-1000.11.45.2)
• [ ] Please attach verbose build log as gist

see https://github.com/Homebrew/homebrew-core/issues/29176#issuecomment-398656987

Temporary workaround:

CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install ...

[joshfriend]

I just upgraded to Mojave and had the same problem. The commandline tools headers package fixed it, but it feels like a hack

[DC-ing]

I have the same problem, but I can solve it after watch wiki

[benmezger]

Solved it by running: sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

[codezakh]

Apparently this is happening because Xcode Command Line tools no longer installs needed headers in /include. You have to run a separate command to install the needed headers.

See https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes.

The command line tools will search the SDK for system headers by default. However, some software may fail to build correctly against the SDK and require macOS headers to be installed in the base system under /usr/include. If you are the maintainer of such software, we encourage you to update your project to work with the SDK or file a bug report for issues that are preventing you from doing so. As a workaround, an extra package is provided which will install the headers to the base system. In a future release, this package will no longer be provided. You can find this package at:
/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

To make sure that you're using the intended version of the command line tools, run xcode-select -s or xcode select -s /Library/Developer/CommandLineTools after installing.

[joshfriend]

I've added this to the suggested build environment and common build problems wiki pages for now until a better solution is found

[qianxiaowei]

I'm now working on Mojave, and followed the solution provided by @thetylerwolf , and it works.

I'm running OS X Sierra and the above didn't help for me. Found this solution here: https://medium.com/@pimterry/setting-up-pyenv-on-os-x-with-homebrew-56c7541fd331
and it did work.

CFLAGS="-I$(brew --prefix readline)/include -I$(brew --prefix openssl)/include -I$(xcrun --show-sdk-path)/usr/include" \
LDFLAGS="-L$(brew --prefix readline)/lib -L$(brew --prefix openssl)/lib" \
PYTHON_CONFIGURE_OPTS=--enable-unicode=ucs2 \
pyenv install -v 3.6.0

[chenguanda]

I'm now working on Mojave too, followed the solution provded by @Voleking, I fixed the error of zlib, but not openssl ,

ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?

I fix it by run the following command

CFLAGS="-I$(brew --prefix openssl)/include" LDFLAGS="-L$(brew --prefix openssl)/lib" pyenv install 3.6.0

[zanderzhng]

I didn't have CLT installed in my Mojave so I couldn't install the headers pkg.
@qianxiaowei 's code worked for me, but I'm wondering if following commands would be enough. below code are enough.

CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install -v 3.7.0

[ssbarnea]

Indeed, the wiki page is misleading as it contains commands that would not work with current released versions of Mojave and xcode 10 but the simple command above works perfects.

Please fix this so we don't need to manually hack CFLAGS to amke it work.

[joshfriend]

@ssbarnea you are always welcome to edit the wiki if you think it is incorrect.

[ssbarnea]

Considering that from now on we will continue to see this problem more I think it would be smart to make pyenv define the CFLAGS itself when is not already defined. This should sort the problem for the vast majority of the users.

For me the problem is sorted as I added it to my ~/.profile alongside other similar hacks related to compilation under MacOS. Still, I do not expect this do be easily doable by every pyenv MacOS user. The best is when it just works ;)

I know that some could argue that this should be done by brew itself, but I really doubt brew would do such a change soon enough to make a difference. By the time they will do, we would probably see 100+ more complaints here, or even worse new duplicate bugs.

[DrewHoo]

Can we link #454 to this issue? It is the exact same problem, but the solution differs if you're on Mojave.

[fharper]

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target / solved it for me, thanks @Voleking .

[tommyjcarpenter]

@Voleking solution also worked for me on a fresh copy of OSX 10.14.1

[moranmcsquid]

@Voleking Would you mind also adding xcode-select --install to your solution.

For most people that would be a "duh", but there are conditions in which you could have installed XCode and have a compiler without having the commandline tools installed.

[tshu-w]

@moranmcsquid added it, thx.

[chikaram]

I had a clean installed Mojave, installed Xcode, Homebrew, pyenv, zlib and still had this issue.

zipimport.ZipImportError: can't decompress data; zlib not available

That means the problem isn't fixed yet at the moment. (Dec 19, 2018)

Anyway, super thx to @Voleking. Your solution perfectly worked for me.

[tarrex]

Well, if you use the brew, zlib not available means that pyenv compilers can't find zlib header files. The brew has told you in brew info zlib that

For compilers to find zlib you may need to set:
export LDFLAGS="-L/usr/local/opt/zlib/lib"
export CPPFLAGS="-I/usr/local/opt/zlib/include"

so just execute

export LDFLAGS="-L/usr/local/opt/zlib/lib"
export CPPFLAGS="-I/usr/local/opt/zlib/include"

and then

pyenv install 3.7.1

There is another warning WARNING: The Python sqlite3 extension was not compiled. Missing the SQLite3 lib? also like this, execute

export LDFLAGS="-L/usr/local/opt/zlib/lib -L/usr/local/opt/sqlite/lib"
export CPPFLAGS="-I/usr/local/opt/zlib/include -I/usr/local/opt/sqlite/include"

then install your version. Worked for me.

• macOS Mojave 10.14.1
• pyenv 1.2.8

[pradyunsg]

@joshfriend Thoughts on detecting this case early, in pyenv install's code and then calling it out before calling into python-build or maybe handling this case at the python-build level?

It could also suggest remedies or maybe try to use the brew zlib if possible; like readline?

[arnavb]

@tarrex Thanks for your solution! It worked perfectly for me.

[iamalisajid]

CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install 3.5.2 worked for me on MACOS Mojave.

[Cabalist]

@iamalisajid This worked for me when the others did not. Thank you

[ssbarnea]

Instead of fixing the wiki, lets try to make pyenv bit smarter and add the missing part to CFLAGS at runtime, this would provide a much better user experience.

[joshfriend]

I don't have lots of time right now, so a pull request adding CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" on macOS would be much appreciated.

[jeffreyzh]

thx to @Voleking. Your solution perfectly worked for me

[imirzadeh]

For me the problem was binutils and I had to uninstall it using:
brew uninstall --force binutils

[jasonmp85]

Hey so why isn't this fixed after, I don't know, four months?

I prefer shell scripts and Ruby and only use Python under necessity. The dependency/environment/version management systems are an inscrutable rat king, so I generally try to keep to best practices by using pipenv and pyenv to keep things from totally borking my system, but I just needed to install a new Python version today and it doesn't Just Work™.

Anyways, doesn't give a great impression.

[Sajjon]

Just wanna confirm what @moranmcsquid said, I had Xcode 10 installed but apparantly this was not enough, the command:

xcode-select --install

Adds the folder Packages in /Library/Developer/CommandLineTools, I was then able to run:

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

As suggested by @benmezger

And then pyenv install 3.7.1 worked.

Apparently it was a stupid assumption to make that being a Swift developer with Xcode 10 installed would have installed what xcode-select --install adds... 🤷‍♂️

[wQueryLoveMm]

Solved it by running: sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

[voctex]

Solved it by running: sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

Thx!

[kylemart]

Sorry if this doesn't add much to the conversation, but @Voleking 's solution worked for me! Super happy 😄

[huntermatthews]

@Volking's solution worked for me as well, however.
I understand that pyenv isn't (and shouldn't) be linked to brew alone. However, if brew is available can we get pyenv install to prefer the brew'd zlib/sqlite/etc just as it does for other things?

Apple is being very ... apple here.

[truegate]

in macOS Mojave 10.14.3 and python 3.7.1, i use xcode-select --install to fix this bug

[pau-lo]

@iamalisajid This worked for me when the others did not. Thank you!!!

[kieran-flaherty]

@tarrex 's sugguestion worked for me, thanks!

[channprj]

Install Python 3.7.2 failed on macOS Mojavi 10.14.4. I have to install macOS SDK manually and It worked.

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

Reference

• Stack Overflow - brew install zlib-devel on Mac OS X Mavericks

[beebird]

This command works on my Mojave:

sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

[slhck]

That's the same command that was posted 6 days ago, which marked as outdated for some reason. It's the only solution that works for me, so I'd prefer keeping that around.

Edit: Well, I see it's part of the original question now, so perhaps not really needed here.

[oatsandsugar]

This article has a workaround --

https://medium.com/@pimterry/setting-up-pyenv-on-os-x-with-homebrew-56c7541fd331

CFLAGS="-I$(brew --prefix readline)/include -I$(brew --prefix openssl)/include -I$(xcrun --show-sdk-path)/usr/include" \ LDFLAGS="-L$(brew --prefix readline)/lib -L$(brew --prefix openssl)/lib" \ PYTHON_CONFIGURE_OPTS=--enable-unicode=ucs2 \ pyenv install -v 2.7.11

[joshfriend]

@slhck I keep marking these as outdated because the correct way to fix the problem is to add CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" to the pyenv install command. You will have to do it manually until we finalize a solution like #1274.

Installing the legacy sdk components works, but is not a good solution long term, so please stop posting it from now on. Thanks :)

[synchronizing]

I didn't have CLT installed in my Mojave so I couldn't install the headers pkg.
@qianxiaowei 's code worked for me, but I'm wondering if following commands would be enough. below code are enough.

CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install -v 3.7.0

Worked with pyenv 1.2.9 installing Python 3.7.2 on Mojave 10.14.4. Fix proposed on #1274, but needs testing.

[Clee681]

As https://github.com/pyenv/pyenv/issues/1219#issuecomment-428700763 points out, a simpler alternative to setting the CFLAGS is to install the system headers manually via:

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Then, the pyenv install commands should just work.

The caveat of this approach is that this extra package may be removed in future releases.

Additional Reference: https://github.com/neovim/neovim/issues/9050#issuecomment-424417456

[slhck]

@Clee681 This has been posted previously; the steps can also be automated (see some comments marked as "outdated" in this thread).

@joshfriend Instead of marking these comments as outdated, it'd probably be better to let them stay visible so people won't post the same solution over and over. Sure, it's not a good workaround, but it's one of the few options available (and, IMHO, the easier one) until there's a more permanent fix.

[joshfriend]

Please keep the conversation on-topic, I want to avoid locking so we can continue to have helpful discussion.

[mamboer]

I didn't have CLT installed in my Mojave so I couldn't install the headers pkg.
@qianxiaowei 's code worked for me, but I'm wondering if following commands would be enough. below code are enough.

CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install -v 3.7.0

Worked with pyenv 1.2.9 installing Python 3.7.2 on Mojave 10.14.4. Fix proposed on #1274, but needs testing.

Works on Mojave 10.14.4 with pyenv install 3.7.3

[slhck]

@HaoweiCh Please read https://github.com/pyenv/pyenv/issues/1219#issuecomment-482699830 — one of the project members believes that it's better to hide this solution, which leads to the fact that other users cannot find it, then post it again, etc. While I completely disagree with this approach, it's at their discretion. Let's just hope that a fix will be out soon.

[joshfriend]

I had been marking the reposts of that solution as spam because they were in my view spam. If a solution has already been posted, people should leave a :+1: to indicate that it helped them instead of continually reposting the same thing and adding noise to the thread. The legacy headers pkg install is the current solution listed in the common build problems wiki as well.

Since hiding the duplicates is still causing confusion, I've unhidden them.

[HaoweiCh]

@joshfriend sorry, I didn't read this issue carefully. and I've already deleted the not that proper solution. :)

[adithyabsk]

Just sum up the current status of this issue, it seems like there is a current solution that breaks tests that @joshfriend has worked on. Just wanted to bump this issue to see where it stands on the docket among other issues as it breaks installation scripts.

[joshfriend]

Big thanks to @mikeroll for coming up with a solution that didn't break the entire test suite like mine did 😆

[Croydon]

Now we only seem to need a new release of pyenv 😄

[Zeesy]

Solved it by running: sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

This worked for me, but only after running
export PATH="$PATH:/Library/Frameworks/Python.framework/Versions/3.7/bin"
(see this post)

[keisuke-nakata]

Finally, my workaround is:

CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl)" CFLAGS="-I$(brew --prefix zlib)/include -I$(brew --prefix sqlite)/include" LDFLAGS="-L$(brew --prefix zlib)/lib -L$(brew --prefix sqlite)/lib" pyenv install 3.7.3

If you have a trouble with ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib? error even when you added LDFLAGS="-L$(brew --prefix openssl)/lib" and CFLAGS="-I$(brew --prefix openssl)/include" .

[rickmark]

One can make this a little easier by adding that to ENV as PYTHON_CONFIGURE_OPTS. Using these env variables (which fix a lot of missing packages on 11.0) one can then simply pyenv install 3.8.5

# added to .zshrc

export PYTHON_CONFIGURE_OPTS="--with-openssl=$(brew --prefix openssl)"
export CFLAGS="-I$(brew --prefix zlib)/include -I$(brew --prefix sqlite)/include -I$(brew --prefix bzip2)/include"
export LDFLAGS="-L$(brew --prefix zlib)/lib -L$(brew --prefix sqlite)/lib -L$(brew --prefix bzip2)/lib"

[ben-xD]

Although this issue is old, the same error message happens on macOS Big Sur.

Just wanted to duplicate what Zheaoli wrote in a different thread:

LDFLAGS="-L$(xcrun --show-sdk-path)/usr/lib" pyenv install 3.8.6 this is working for me

Using /lib instead of /include as thread-OP allowed me to install 3.8.6.