Pipenv: Cannot install under Ubuntu 18.04, breaks pip ("ImportError: cannot import name main")

[slhck]

I want to install pipenv under Ubuntu 18.04. When I do so, it breaks pip / pip3.

Expected result

Installed and working version of pipenv.

Actual result

pip / pip3 are broken, depending on whether I wanted to install pipenv though pip or pip3.

➜ pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

Steps to replicate

1. Set up Ubuntu 18.04
2. Run pip install pipenv or pip3 install pipenv
3. Run pip or pip3 – the error is printed, and pip / pip3 do not work anymore.

In order to fix the issue I have to run:

sudo python -m pip uninstall pip && sudo apt install python-pip --reinstall
sudo python3 -m pip uninstall pip && sudo apt install python3-pip --reinstall

But I cannot install pipenv using pip. Installation through apt does not work because there is no PPA available..

---

Solution

See the solution below; you need to have

export PATH="${HOME}/.local/bin:$PATH"

in your shell configuration. If the path is not there, it won't work.

[slhck]

Here is the output from pip3:

werner in ~ at octopus23
➜ pip3 install pipenv
Collecting pipenv
  Using cached https://files.pythonhosted.org/packages/2c/01/37a5867a47d52856b077d0faa561b791cb6e6e3e9410837b6d62f569c1e6/pipenv-11.10.1-py3-none-any.whl
Collecting virtualenv (from pipenv)
  Using cached https://files.pythonhosted.org/packages/ed/ea/e20b5cbebf45d3096e8138ab74eda139595d827677f38e9dd543e6015bdf/virtualenv-15.2.0-py2.py3-none-any.whl
Collecting pip>=9.0.1 (from pipenv)
  Using cached https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl
Collecting setuptools>=36.2.1 (from pipenv)
  Using cached https://files.pythonhosted.org/packages/8c/10/79282747f9169f21c053c562a0baa21815a8c7879be97abd930dbcf862e8/setuptools-39.1.0-py2.py3-none-any.whl
Collecting virtualenv-clone>=0.2.5 (from pipenv)
  Using cached https://files.pythonhosted.org/packages/6d/c2/dccb5ccf599e0c5d1eea6acbd058af7a71384f9740179db67a9182a24798/virtualenv_clone-0.3.0-py2.py3-none-any.whl
Collecting certifi (from pipenv)
  Using cached https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl
Installing collected packages: virtualenv, pip, setuptools, virtualenv-clone, certifi, pipenv
Successfully installed certifi-2018.4.16 pip-10.0.1 pipenv-11.10.1 setuptools-39.1.0 virtualenv-15.2.0 virtualenv-clone-0.3.0

[uranusjr]

I’m guessing this is likely a consequence of Pipenv depending on pip, but I’m not sure if Pipenv should be responsible for this. Maybe Debian should because it’s them who broke the Python-pip coupling. Pipenv is certainly not the only package out there depending on pip either, but I’m not sure what is the best practice here.

@ncoghlan could you provide some insight on this topic? Specifically

1. Should a package depend on pip?
2. If 1. is yes, should it be responsible for this kind of conflict with system package manager?
3. If 2. is yes, how?
4. If 2. is no, who should be responsible? Should the user be instructed to use a workaround instead?

[uranusjr]

In any case, you probably should never sudo pip install anything on Ubuntu anyway. You should do one of the followings instead

• Use APT all the way down
• pip install --user
• A custom manager such as pipsi

Or even better, avoid system Python altogether, and use pyenv or other Python runtime managers instead.

Scratch those. Avoid Python from APT, period.

[slhck]

you probably should never sudo pip install anything on Ubuntu anyway

I actually never did. I'm aware of the problems of using sudo pip under Ubuntu, so I use apt whenever possible or stick to --user.

[uranusjr]

Ah I see the problem now. User modules take precedence over system ones, but /usr/bin is before $HOME/.local/bin in you PATH. /usr/bin/python tries to import the system pip installation (which is still 9.x), but ended up finding the user installation (which is 10.x). What a mess.

[uranusjr]

2095 is the same, but I want to keep this open because I think there should be a more robust solution than to depend on the user having a friendly PATH.

[gsemet]

it is pretty stable when user has :

• ˜/.local/bin at the begining of user's PATH. Should be the default on Ubuntu, since 16.10. It is a good practice to have it since it enables pip install --user behavior that is a perfectly valid use case
• user should always use pip install --user. Messing with the system's pip is really bad in every distribution, even the weird "dist-packages/site-packages" folders in debian do not protect against mistakes from user.

We have ensured this behavor in all our ubuntu user installs (organisation of more that 1000 installs of various ubuntu version), and the transition to pip10 worked like a charm.

[techalchemy]

While I agree that python and its ecosystem is complicated and that it would be better if people didn't have to properly configure their paths to get the search order right, this is unfortunately just a fact about the way installation works right now in package managers and it's not exactly a pipenv issue per se. I don't think there is much we can do to independently tackle it, it's more of an issue to be addressed on the python mailing lists

[slhck]

Ah, I see now what the problem is, thanks. Never thought that the path could influence this, which is why it didn't occur to me to include it in the issue description. Sorry about that, and thanks for your help.

What fixed it for me was adding

export PATH="${HOME}/.local/bin:$PATH"

to the profile.

Edit: Make sure you run hash -r or enter a new shell for this change to take effect.

I don't think there is much we can do to independently tackle it

Perhaps there could be some more detailed installation instructions or caveats? I'm not too experienced with how pip works, but I might have read a note about path issues. But I guess that the ecosystem of different package managers and distributions is too complex for a simple rule…

[techalchemy]

@slhck it's actually so frustrating that there was this xkcd specifically about python environments and sudo and package manager installations a few days ago

[jacobmbr]

Here's a nice step-by-step guide I used successfully on Ubuntu 18.04:
https://phoikoi.io/2018/04/03/bootstrap-pipenv-debian.html

[ncoghlan]

There's also https://github.com/pypa/python-packaging-user-guide/issues/396, which discusses the question of whether or not we might be able to come up with the checklist or troubleshooting script to help folks identify and resolve potential issues in their environment. I'll make a note there about the potential for PATH/sys.path ordering conflicts.

[texadactyl]

Thanks, @slhck . Your export work-around saved me the most time; I've added it to /etc/profile.

It's truly dumb that we have 2 levels of software packaging on Linux. Even worse, with the latest release of the *buntus, both flavors of one of them (pip/pip3) is broken.

I wrote it up on Launchpad: https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1772746

[uranusjr]

@texadactyl Debian has this policy (I forget what it is) that pip shouldn’t be installed by default, and they’ve stuck to it with much previous complaint. The tradition lays deep, deep in Debian roots. I’d be trilled if they can rethink this, but I highly doubt they would.

[uzzzval]

I was facing the same issue on Ubuntu 18 and Python 3.6

Below are the steps i followed:

1) Firstly i was receiving the error :

Traceback (most recent call last):
File "/usr/bin/pip", line 9, in
from pip import main
ImportError: cannot import name main

2) I modified /user/bin/pip file to:

import sys
from pip._internal import main as _main
if __name__ == '__main__':
sys.exit(main())

3) Then, it started giving me this error:

Traceback (most recent call last):
File "/usr/bin/pip3", line 11, in
sys.exit(main())
NameError: name 'main' is not defined

4) I modified /usr/bin/pip3 to:

import sys
from pip._internal import main as _main
if __name__ == '__main__':
sys.exit(_main())

5) Then i started getting the error:

Traceback (most recent call last):
File "/usr/bin/pip3", line 11, in
sys.exit(main())
NameError: name 'main' is not defined

6) I renamed main() to _main(), and voila .. it worked !!! :) :)

[slhck]

While that may have solved the problem for you, this is quite bad advice. You should not manually modify system files. Please have a look at my earlier comment for a solution.

[ConorSheehan1]

edit: @slhck 's solution solved it. Need to have ~/.local/bin in your path

Getting the same problem on ubuntu 16.04.

$ sudo apt install python3-pip
$ pip3 --version
pip 8.1.1 from /usr/lib/python3/dist-packages (python 3.5)
$ python3 -m pip install --user pipenv
$ pip3 --version
Traceback (most recent call last):
  File "/usr/bin/pip3", line 9, in <module>
    from pip import main
ImportError: cannot import name 'main'

# revert back and fix pip
$ sudo python3 -m pip uninstall pip && sudo apt install python3-pip --reinstall

[thernstig]

I just want to add that I stumbled upon this issue as well. I already had ~/.local/bin as the first thing in my path. The problem is how bash hashes commands. The solution to this problem was found here: https://github.com/pypa/pip/issues/5221#issuecomment-381568428

[slhck]

@thernstig Yes, that's correct – I added the necessary hash -r command to my solution above, which I forgot to explicitly mention.

[thernstig]

@slhck No problem, glad I could help

[jlitzingerdev]

I realized the issue is closed, but I'm posting with the hope this might help avoid modifications to PATH. I ran in to this today when setting up a new Ubuntu 18.04 machine and was able to resolve it without PATH modifications, though I did reboot (I'm fairly certain log out/in will work, but didn't verify).

After installing pip3 via python3-pip and pipenv with pip3 install --user pipenv I received the subject error. I was about to use the workaround from @slhck when I noticed the following in ~/.profile (stock, I have no modifications):

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi

Curious, I rebooted and sure enough, the issue was resolved as my .local/bin was now at the beginning of my PATH and pip3 worked again.

[slhck]

@jlitzingerdev I'm pretty sure that path is the default, it's just that for my system, since I was using a different shell and profile, I removed it or constructed it from scratch, hence my “workaround” which should not have been necessary in the first place. Glad you got it figured out.

[jlitzingerdev]

@slhck It is, but it may not be in PATH on the first login because ~/.local/bin may not exist, or such was my specific case.

[GreatBahram]

What do we have to do?

[ChadBailey]

Just want to say thank you for including restoration commands, I was having trouble getting pip to work again.

[r-tron18]

I want to install pipenv under Ubuntu 18.04. When I do so, it breaks pip / pip3.

Expected result

Installed and working version of pipenv.

Actual result

pip / pip3 are broken, depending on whether I wanted to install pipenv though pip or pip3.

➜ pip
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    from pip import main
ImportError: cannot import name main

Steps to replicate

1. Set up Ubuntu 18.04

2. Run `pip install pipenv` or `pip3 install pipenv`

3. Run `pip` or `pip3` – the error is printed, and pip / pip3 do not work anymore.

In order to fix the issue I have to run:

sudo python -m pip uninstall pip && sudo apt install python-pip --reinstall
sudo python3 -m pip uninstall pip && sudo apt install python3-pip --reinstall

But I cannot install pipenv using pip. Installation through apt does not work because there is no PPA available..

i have 3 pip in my ubuntu 18 pip, pip3 and pip3.6 pip is for 2.7 pip3 is for 3.5 and pip3.6 for 3.6 now which pip shows the file location in .local/bin . i removed the file pip and pip3 from here. now which pip3 shows me the /usr/bin/pip3. run the command sudo nano /usr/bin/pip3 change the first line for interpreter as !#/usr/bin/python3 to #/usr/bin/python3.5. works for me. all my pips work. i hope this helps

[slhck]

i have 3 pip in my ubuntu 18 pip, pip3 and pip3.6 pip is for 2.7 pip3 is for 3.5 and pip3.6 for 3.6 now which pip shows the file location in .local/bin . i removed the file pip and pip3 from here. now which pip3 shows me the /usr/bin/pip3. run the command sudo nano /usr/bin/pip3 change the first line for interpreter as !#/usr/bin/python3 to #/usr/bin/python3.5. works for me. all my pips work. i hope this helps

All these steps are really not encouraged. You should not manually edit files in /usr/bin. Instead, try to uninstall pip as mentioned above, and reinstall via apt. This should get you the correct system installations. If you then have problems with pipenv, please open a new issue and describe your problem rather than attempting such workarounds.

[r-tron18]

reinstalling via apt as mentioned above will not change anything as it generates the same output. It simple states cannot import name main and on doing import pip in a python interpreter it does import it as module , means the name main is not available in the module where it is trying to look. Now i didnot suggest above edit blindly. As a software enthusiast when i was on ubuntu14 i checked pip file and it has the same code as the one in ubuntu18 pip3 . so what changed. The python version that comes prepackaged with it did.

Because pip for python3.5 and 2.7 has only main defined in pip module.
And python3.6 has it defined in pip._internal

Now the above two statement solves everyones problem

the problem lies in the interpreter used itself

python3; but which python is it referencing to
Developers who had only python 2.7 and changed to ubuntu18 will likely have python 3 as python3.6

but what about those who had python3.4 pr 3.5 installed. First the python3 referenced to python3.5 and now after upgrade it will refer to ppython3.6

So the way it was handled by debian broke it

merely stating the correct interpreter will solve the problem
of both having errors

from pip import main
and
from pip._internal import main as _main

Thanks
ps
At this point i must say that lecturing what not to will be ok on other platform ,but on github where other passionate developers come to solve there nuisances ; specially when they are trying to run a lots of commands or edit some config files to get it going, i must say dont be so smart that you start discouraging people like this.

NOTE: The new versions aren't going to work if they don't change the code. They will add workarounds so that it works for everyone uninstalling and installing it

I hope at this point you will get it. Peace and don't bother me again

[slhck]

@r-tron18 I am sorry you feel lectured by what was meant as a constructive suggestion — to not have you modify a system file. From the little context you were giving in your original post, it was impossible to tell whether you're an experienced user who knows what they're doing, or just applying workarounds they might have found elsewhere. I spend a lot of time on various troubleshooting and Q&A websites, and I guess you'd agree that we should not encourage users to try random fixes or sudo-edit a system-shipped file, which often leads to more errors and confusion. That is more of a general issue.

I am still not convinced that what you are suggesting is a reliable solution. And: we can have a civil discussion about the merits of that solution — GitHub is a place where I'm allowed to express that.

The reason is: changing the shebang of /usr/bin/pip3 to #!/usr/bin/python3.5 is only going to get you stuck with that version. It should read #!/usr/bin/python3, and that should be a symlink to /usr/bin/python3.6 (or whatever is current on your system). Any Python upgrade will likely change that file anyway.

You say:

but what about those who had python3.4 pr 3.5 installed. First the python3 referenced to python3.5 and now after upgrade it will refer to ppython3.6
So the way it was handled by debian broke it

Are you saying that after an upgrade of Python 3.5 to Python 3.6, that python3 symlink was not updated? If what you are observing is indeed a bug with Debian or Ubuntu, where a Python upgrade failed to set correct symlinks, then that bug should probably be addressed upstream, no?

[JulioCoder]

I realized the issue is closed, but I'm posting with the hope this might help avoid modifications to PATH. I ran in to this today when setting up a new Ubuntu 18.04 machine and was able to resolve it without PATH modifications, though I did reboot (I'm fairly certain log out/in will work, but didn't verify).

After installing pip3 via python3-pip and pipenv with pip3 install --user pipenv I received the subject error. I was about to use the workaround from @slhck when I noticed the following in ~/.profile (stock, I have no modifications):

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi

Curious, I rebooted and sure enough, the issue was resolved as my .local/bin was now at the beginning of my PATH and pip3 worked again.

It works for me, after rebooted all work fine. (after install: "pip3 install --user pipenv" restart your system and it will work.