Pyenv: After pip 10 upgrade on pyenv "ImportError: cannot import name 'main'"

Created on 15 Apr 2018 · 24Comments · Source: pyenv/pyenv

pyenv 1.1.3
Pip version: 10.0
Python version: 3.6.2
Operating system: Ubuntu 16.04

Description:

After upgrading pip from 9.03 to 10.0 via pip install pip --user --upgrade in a pyenv environment pip refuses to start and raise this instead:

Traceback (most recent call last):
  File "/home/kleinernull/.pyenv/versions/3.6.2/bin/pip", line 7, in <module>
    from pip import main
ImportError: cannot import name 'main'
Traceback (most recent call last):
  File "/home/kleinernull/.pyenv/versions/3.6.2/bin/pip", line 7, in <module>
    from pip import main
ImportError: cannot import name 'main'

The content of all three different pip files is the same:

~ ⟩ cat .pyenv/versions/3.6.2/bin/pip                                                                            ~
#!/home/kleinernull/.pyenv/versions/3.6.2/bin/python3.6

# -*- coding: utf-8 -*-
import re
import sys

from pip._internal import main as _main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(_main())

~ ⟩ cat .pyenv/versions/3.6.2/bin/pip3                                                                           ~
#!/home/kleinernull/.pyenv/versions/3.6.2/bin/python3.6


# -*- coding: utf-8 -*-
import re
import sys

from pip import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

~ ⟩ cat .pyenv/versions/3.6.2/bin/pip3.6                                                                         ~
#!/home/kleinernull/.pyenv/versions/3.6.2/bin/python3.6

# -*- coding: utf-8 -*-
import re
import sys

from pip import main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

The same happens with my 3.6.1 environment too.

Temporally fix

According to the code of the master branch the import should be that:

#!/home/kleinernull/.pyenv/versions/3.6.2/bin/python3.6

# -*- coding: utf-8 -*-
import re
import sys

from pip._internal import main as _main

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(_main())

And this resolves the issue. I have no clue if this has something to do with the upgrade itself or with pyenv as environment.

Source

KleinerNull

👍4 ❤1

Most helpful comment

The following fixed it for me: python -m pip install --force-reinstall pip

tekumara on 5 Jun 2018

👍16 ❤14 🎉11

All 24 comments

Note that importing modules from pip is not supported - see https://pip.pypa.io/en/latest/user_guide/#using-pip-from-your-program

In particular importing pip._internal.main is not supported (hence the use of _internal in the name) - the correct solution as noted in the docs is to call pip as a subprocess.

pfmoore on 15 Apr 2018

👎6 😕3 👍1

@pfmoore

Ok, so what caused this in the first places, I just upgraded pip inside the env with pip install pip --user --upgrade and it looks like that the script didn't changed. This was just a try to fix the ImportError.

Are these files created by pyenv or by pip?

Besides not being supported it works, but I will change it to the subprocess way.

KleinerNull on 16 Apr 2018

In general, pyenv doesn't care the packages installed within specific Python version. It sometime manages some symlinks (e.g. pip3 -> pip) to make shim script works better with pyenv, though. Totally this is some issue in pip itself, not pyenv.

yyuu on 16 Apr 2018

@yyuu

Totally this is some issue in pip itself, not pyenv.

Funny, I've created first an issue over at the pip repo and the first respone was that this is a pyenv issue so I create here the same too.

KleinerNull on 16 Apr 2018

😄12 👍2

Hi @yyuu!

Totally this is some issue in pip itself, not pyenv.

What makes you say so? Because the shim script failed? That failed because, as @pfmoore notes, the script is using the internals of pip, which can be (and, in this case, have) changed.

How exactly is this shim script being generated? I guessed that pyenv was generating the shim and hence redirected @KleinerNull here.

pradyunsg on 16 Apr 2018

I had no issue with pip here, even using pyenv. I always run pyenv rehash after most of the operations ran with pyenv, so I didn't even noticed this problem.

vltr on 20 Apr 2018

I know little or nothing about pyenv, as I'm a Windows user not a Linux user, but from what I can see, pyenv creates "shims" for the various executables, including pip. The issue here looks very like running a pip 9 executable wrapper against a pip 10 installation (which won't work, and indeed will fail exactly as @KleinerNull is seeing). I can only surmise, but if you run pip install --upgrade pip in a pyenv environment, that will update pip, but won't fix the shim, and that will behave as you are seeing. The solution would then be, as suggested by @vltr, to run pyenv rehash.

Apologies for implying that this was an issue with pyenv itself - it now appears that it may be simply that the user missed a pyenv-specific step needed when upgrading a copy of pip that's in a pyenv environment.

pfmoore on 20 Apr 2018

👍1

I don't have problems with install (I have pyenv and virtualenvs for 2.7, 3.6 and pypy 2.7 projects) but my recommendation is that pyenv install by default latest pip 9.x version, not pip 10, because many products will faill by the API changes. For example I have deployed some programs in AWS-Lambda with Zappa and we have now a compatibility problem by the different API, check this, this and this

ysolis on 23 Apr 2018

This is because upgrading pip breaks debian's executable pip installed via python-pip.

There's a list of workarounds in https://github.com/pypa/pip/issues/5221#issuecomment-382069604. This issue can be closed.

pradyunsg on 24 Apr 2018

👍4

Thanks @pradyunsg

in this world nothing can be said to be certain, except death, taxes and Debian packaging nonsense

joshfriend on 24 Apr 2018

😄2

Hey guys, I was brought here by a reference from an issue over at zappa. I'm genuinely curious but is this issue not related to the fact that pip has changed their library and got rid of pip.main() for pip._internal? The use of from pip import main and some other well used methods were considered unsupported and have been changed in pip>=10.0.0 and from some reports have even been broken in pip==9.0.2.

Take a peek at the following issue. It's a mess fucking mess. The maintainers of pip suggest using subprocess instead of pip.main().

https://github.com/pypa/pip/issues/5081

iamjohnnym on 24 Apr 2018

The following fixed it for me: python -m pip install --force-reinstall pip

tekumara on 5 Jun 2018

👍16 ❤14 🎉11

python -m pip install --force-reinstall pip

this didn't work for me.

stevehemingway on 6 Jun 2018

Hi all, having the same issue. I have tried completely reinstalling pip through apt and through @tukushan 's suggestion:

python -m pip install --force-reinstall pip

Has anyone found a way to resolve this issue?

michaelpitts on 7 Jun 2018

I have reinstalled pip and pip-whl, then ran @tukushan suggestion and it worked.

Ubuntu 16.04.3

wesleyegberto on 17 Jun 2018

@tukushan's workaround didn't work for me, so after series of unsucsessfull reinstalls I've tried this workaround in my Ubuntu 16.04.4 LTS:

~$ pip
bash: /usr/bin/pip: No such file or directory
~$ which pip
/usr/local/bin/pip
~$ /usr/local/bin/pip

Apparently, after upgrade, pip installed itself into different directory, so I've just symlinked it to previous path. So far so good.

~$ sudo ln -s /usr/local/bin/pip /usr/bin/pip
~$ pip --version
pip 10.0.1 from /usr/local/lib/python2.7/dist-packages/pip (python 2.7)
~$ pip

Usage:   
  pip <command> [options]

Commands:
  install                     Install packages.
  download                    Download packages.
...

This is not official fix.

apelsina on 25 Jun 2018

❤6

@apelsina's solution works for me in my Ubuntu 16.04.4 LTS

jonathantzh on 25 Jun 2018

Downgrade pip 10 version to pip 9.0.3 by
sudo python -m pip install pip==9.0.3 --upgrade
This worked for me in my Ubuntu 16.04.4 LTS and Python 2.7 version.

tejasri19 on 10 Jul 2018

❤6

I getting the same error:

pip uninstall scikit-image

Traceback (most recent call last):
File "/usr/bin/pip", line 9, in
from pip import main
ImportError: cannot import name main

python 2.7.12
pip 18.0
ubuntu 16.04

zkhodzhaev on 23 Jul 2018

👍1

I'd like to point out that pypa/pip#5599 talks about this issue in some detail. If you're getting an ImportError while using pip, I request you to take a look at that issue and those linked from it.

pradyunsg on 23 Jul 2018

Since so many have come to rely on pip.main (understanding that it was never intended to be used) and since the docs state that the following is supported:

subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'my_package'])

wouldn't it make sense to add in official support for pip.main with something like the following:

def main(*args):
    subprocess.check_call([sys.executable, '-m', 'pip', *args])

This would allow people to continue to use the same pip.main functionality in a supported way while still allowing the pip devs to hide away the non-public interface as an implementation detail.