pipenv install of pycurl produces error with link-time ssl backend vs compile-time ssl backend

Created on 9 Jan 2018 · 4Comments · Source: pypa/pipenv

Issue

When using pipenv install to install pycurl the following error is produced when attempting to use pycurl.

ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)

For the system python3 version the following was used to install pycurl

pip3 install --install-option="--with-openssl" pycurl

From reading:

It wasn't obvious where I could add the "--with-openssl" for pipenv to pick up as part of the install flags.

I did attempt to add a pip.conf with the following entry

[global]
with-openssl = true

It did not work but I could have been using that incorrectly. I had tried that based on the posting at https://stackoverflow.com/questions/21096436/ssl-backend-error-when-using-openssl

Describe your environment

MacOS X High Sierra (version 10.13.2)
System Python versions:
> python -V
> Python 2.7.14
> python3 -V
> Python 3.6.4
Pipenv Python version
> (whois_query-CYnyCQ73)) bash-3.2$ python -V
> Python 3.6.4
Pipenv version: $ pipenv --version
> (whois_query-CYnyCQ73) bash-3.2$ pipenv --version
> pipenv, version 9.0.1

Expected result

With a simple pycurl script of the following

#!/usr/bin/python

import pycurl

c = pycurl.Curl()
c.setopt(c.URL, 'http://www.google.com')
c.perform()

With the system python this returns the content of the main google home page.

<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en"><head><meta content="Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for." name="description"><meta content="noodp" name="robots"><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/logos/doodles/2018/har-gobind-khoranas-96th-birthday-4731112378073088.2-l.png" .....

Actual result

When running the same script in the pipenv shell the following is the result.

(whois_query-CYnyCQ73) bash-3.2$ python -v google_query.py 
import _frozen_importlib # frozen
import _imp # builtin
import sys # builtin
import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
import '_io' # <class '_frozen_importlib.BuiltinImporter'>
import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
import _thread # previously loaded ('_thread')
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import _weakref # previously loaded ('_weakref')
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
# installing zipimport hook
import 'zipimport' # <class '_frozen_importlib.BuiltinImporter'>
# installed zipimport hook
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/__pycache__/__init__.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/__init__.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/__pycache__/__init__.cpython-36.pyc'
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/codecs.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/codecs.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/codecs.cpython-36.pyc'
import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x104942780>
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/__pycache__/aliases.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/aliases.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/__pycache__/aliases.cpython-36.pyc'
import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x10497e2b0>
import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x1049422e8>
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/__pycache__/utf_8.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/utf_8.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/__pycache__/utf_8.cpython-36.pyc'
import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x1049890b8>
import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/__pycache__/latin_1.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/latin_1.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/encodings/__pycache__/latin_1.cpython-36.pyc'
import 'encodings.latin_1' # <_frozen_importlib_external.SourceFileLoader object at 0x104989b38>
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/io.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/io.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/io.cpython-36.pyc'
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/abc.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/abc.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/abc.cpython-36.pyc'
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/_weakrefset.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/_weakrefset.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/_weakrefset.cpython-36.pyc'
import '_weakrefset' # <_frozen_importlib_external.SourceFileLoader object at 0x104990a58>
import 'abc' # <_frozen_importlib_external.SourceFileLoader object at 0x104990128>
import 'io' # <_frozen_importlib_external.SourceFileLoader object at 0x104989d30>
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/site.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/site.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/site.cpython-36.pyc'
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/os.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/os.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/os.cpython-36.pyc'
import 'errno' # <class '_frozen_importlib.BuiltinImporter'>
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/stat.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/stat.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/stat.cpython-36.pyc'
import '_stat' # <class '_frozen_importlib.BuiltinImporter'>
import 'stat' # <_frozen_importlib_external.SourceFileLoader object at 0x104a1bf60>
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/posixpath.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/posixpath.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/posixpath.cpython-36.pyc'
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/genericpath.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/genericpath.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/genericpath.cpython-36.pyc'
import 'genericpath' # <_frozen_importlib_external.SourceFileLoader object at 0x104a21940>
import 'posixpath' # <_frozen_importlib_external.SourceFileLoader object at 0x104a1c630>
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/_collections_abc.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/_collections_abc.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/_collections_abc.cpython-36.pyc'
import '_collections_abc' # <_frozen_importlib_external.SourceFileLoader object at 0x104a21f60>
import 'os' # <_frozen_importlib_external.SourceFileLoader object at 0x1049a7e10>
# /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/_bootlocale.cpython-36.pyc matches /Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/_bootlocale.py
# code object from '/Users/homedir/.virtualenvs/whois_query-CYnyCQ73/bin/../lib/python3.6/__pycache__/_bootlocale.cpython-36.pyc'
import '_locale' # <class '_frozen_importlib.BuiltinImporter'>
import '_bootlocale' # <_frozen_importlib_external.SourceFileLoader object at 0x104a0c208>
import 'site' # <_frozen_importlib_external.SourceFileLoader object at 0x1049a13c8>
Python 3.6.4 (default, Jan  6 2018, 11:51:59) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
  File "google_query.py", line 3, in <module>
    import pycurl
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 658, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 922, in create_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
# clear builtins._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.__interactivehook__
# clear sys.flags
# clear sys.float_info
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup[2] removing builtins
# cleanup[2] removing sys
# cleanup[2] removing _frozen_importlib
# cleanup[2] removing _imp
# cleanup[2] removing _warnings
# cleanup[2] removing _thread
# cleanup[2] removing _weakref
# cleanup[2] removing _frozen_importlib_external
# cleanup[2] removing _io
# cleanup[2] removing marshal
# cleanup[2] removing posix
# cleanup[2] removing zipimport
# cleanup[2] removing encodings
# destroy encodings
# cleanup[2] removing codecs
# cleanup[2] removing _codecs
# cleanup[2] removing encodings.aliases
# cleanup[2] removing encodings.utf_8
# cleanup[2] removing _signal
# cleanup[2] removing __main__
# destroy __main__
# cleanup[2] removing encodings.latin_1
# cleanup[2] removing io
# destroy io
# cleanup[2] removing abc
# cleanup[2] removing _weakrefset
# destroy _weakrefset
# cleanup[2] removing site
# destroy site
# cleanup[2] removing os
# cleanup[2] removing errno
# cleanup[2] removing stat
# cleanup[2] removing _stat
# cleanup[2] removing posixpath
# cleanup[2] removing genericpath
# cleanup[2] removing os.path
# cleanup[2] removing _collections_abc
# destroy _collections_abc
# cleanup[2] removing _bootlocale
# destroy _bootlocale
# cleanup[2] removing _locale
# destroy zipimport
# destroy _signal
# destroy abc
# destroy errno
# destroy posixpath
# destroy _stat
# destroy genericpath
# destroy os
# destroy stat
# destroy _locale
# cleanup[3] wiping _frozen_importlib
# destroy _frozen_importlib_external
# cleanup[3] wiping _imp
# cleanup[3] wiping _warnings
# cleanup[3] wiping _thread
# cleanup[3] wiping _weakref
# cleanup[3] wiping _io
# cleanup[3] wiping marshal
# cleanup[3] wiping posix
# cleanup[3] wiping codecs
# cleanup[3] wiping _codecs
# cleanup[3] wiping encodings.aliases
# cleanup[3] wiping encodings.utf_8
# cleanup[3] wiping encodings.latin_1
# cleanup[3] wiping sys
# cleanup[3] wiping builtins

Steps to replicate

mkdir pycurl_troubleshooting && cd pycurl_troubleshooting
pipenv install --three

Creating a virtualenv for this project…
Using /usr/local/bin/python3 to create virtualenv…
⠋Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix '/usr/local/Cellar/python3/3.6.4_2/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/homedir/.virtualenvs/pycurl_troubleshooting-q7o1wxXB/bin/python3.6
Also creating executable in /Users/homedir/.virtualenvs/pycurl_troubleshooting-q7o1wxXB/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /Users/homedir/.virtualenvs/pycurl_troubleshooting-q7o1wxXB
Creating a Pipfile for this project…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (625834)!
Installing dependencies from Pipfile.lock (625834)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run the following:
 $ pipenv shell

pipenv shell
python -V

(pycurl_troubleshooting-q7o1wxXB) bash-3.2$ python -V
Python 3.6.4

vi google.py

#!/usr/bin/python

import pycurl

c = pycurl.Curl()
c.setopt(c.URL, 'http://www.google.com')
c.perform()

pipenv install pycurl
python google_query.py

(pycurl_troubleshooting-q7o1wxXB) bash-3.2$ python google_query.py 
Traceback (most recent call last):
  File "google_query.py", line 3, in <module>
    import pycurl
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)

Source

linkn3echo

👍4

Most helpful comment

Neither reinstalling openssl, or running PIP_INSTALL_OPTION="--with-openssl" pipenv install works for me.

The only thing that does allow pipenv install to run is:

pip uninstall pycurl (yes, even though we're using pipenv)
export PYCURL_SSL_LIBRARY=nss (or =openssl may also work)
pip install pycurl
pipenv --rm
pipenv install

lakesare on 27 Sep 2018

👍3

All 4 comments

I found that using this:

PIP_INSTALL_OPTION="--with-openssl" pipenv install works. but that's not really convenient

MichaelBitard on 18 Jan 2018

Thanks MichaelBitard, my apologies for the late response.

@MichaelBitard or @kennethreitz Is there a way to specify the option listed in the pipfile along the lines of the dependency specifications PEP 508.

Something like

[packages]

pycurl = "{ with-openssl }"

Otherwise would I have to tell anyone using the project to run

_"PIP_INSTALL_OPTION="--with-openssl" pipenv install"_

instead of

_pipenv install_, correct?

@kennethreitz was this closed because I did not respond in time?

linkn3echo on 26 Jan 2018

I'm having the same issue.
I've tried using the solution above but still not working.
I solved my issue by reinstall my openssl following this link

$ brew reinstall openssl
$ brew link --force openssl
$ ln -s /usr/local/opt/openssl/include/openssl /usr/local/include

Keda87 on 8 Aug 2018

Neither reinstalling openssl, or running PIP_INSTALL_OPTION="--with-openssl" pipenv install works for me.

The only thing that does allow pipenv install to run is:

pip uninstall pycurl (yes, even though we're using pipenv)
export PYCURL_SSL_LIBRARY=nss (or =openssl may also work)
pip install pycurl
pipenv --rm
pipenv install

lakesare on 27 Sep 2018

👍3

Was this page helpful?

0 / 5 - 0 ratings

Related issues

$ pipenv install not working.

jeyraof · 3Comments

Clarify & document integration with conda

fbender · 3Comments

pipenv fails to lock on first installed package

jacebrowning · 3Comments

dependencies could not be resolved

leileigong · 3Comments

incorrect dependencies (versions) locked by pip

jerzyk · 3Comments