poetry install creates venv using compatible version (python3) but tries to install dependencies using pip v2

[nickperkins]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS Mojave 10.14.6
• Poetry Version: 1.0.0
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/nickperkins/f99285f9ce978f2b561717ae6104d918

Issue

pip is unable to install the required package as it does not target python 2. Once a venv is created it should use the pip installed in the venv to install the packages.

poetry install -vvv         <aws:amazon-web-services>
The currently activated Python version 2.7.17 is not supported by the project (^3.6).
Trying to find and use a compatible version.
Trying python3
Using python3 (3.7.5)
Virtualenv tool-it-02NcbsqG-py3.7 already exists.
Using virtualenv: /Users/nick.perkins/Library/Caches/pypoetry/virtualenvs/tool-it-02NcbsqG-py3.7
Installing dependencies from lock file


Package operations: 28 installs, 0 updates, 0 removals, 2 skipped

  - Installing more-itertools (8.0.2)

[EnvCommandError]
Command ['pip', 'install', '--no-deps', 'more-itertools==8.0.2'] errored with the following return code 1, and output:
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
ERROR: Could not find a version that satisfies the requirement more-itertools==8.0.2 (from versions: 1.0, 1.1, 2.0, 2.1, 2.2, 2.3, 2.4, 2.4.1, 2.5.0, 2.6.0, 3.0.0, 3.1.0, 3.2.0, 4.0.0, 4.0.1, 4.1.0, 4.2.0, 4.3.0, 5.0.0)
ERROR: No matching distribution found for more-itertools==8.0.2


Traceback (most recent call last):
  File "/Users/nick.perkins/.poetry/lib/poetry/_vendor/py2.7/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/Users/nick.perkins/.poetry/lib/poetry/_vendor/py2.7/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/Users/nick.perkins/.poetry/lib/poetry/_vendor/py2.7/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/Users/nick.perkins/.poetry/lib/poetry/_vendor/py2.7/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/Users/nick.perkins/.poetry/lib/poetry/console/commands/install.py", line 63, in handle
    return_code = installer.run()
  File "/Users/nick.perkins/.poetry/lib/poetry/installation/installer.py", line 74, in run
    self._do_install(local_repo)
  File "/Users/nick.perkins/.poetry/lib/poetry/installation/installer.py", line 286, in _do_install
    self._execute(op)
  File "/Users/nick.perkins/.poetry/lib/poetry/installation/installer.py", line 302, in _execute
    getattr(self, '_execute_{}'.format(method))(operation)
  File "/Users/nick.perkins/.poetry/lib/poetry/installation/installer.py", line 327, in _execute_install
    self._installer.install(operation.package)
  File "/Users/nick.perkins/.poetry/lib/poetry/installation/pip_installer.py", line 97, in install
    self.run(*args)
  File "/Users/nick.perkins/.poetry/lib/poetry/installation/pip_installer.py", line 118, in run
    return self._env.run_pip(*args, **kwargs)
  File "/Users/nick.perkins/.poetry/lib/poetry/utils/env.py", line 824, in run_pip
    return self._run(cmd, **kwargs)
  File "/Users/nick.perkins/.poetry/lib/poetry/utils/env.py", line 1052, in _run
    return super(VirtualEnv, self)._run(cmd, **kwargs)
  File "/Users/nick.perkins/.poetry/lib/poetry/utils/env.py", line 856, in _run
    raise EnvCommandError(e, input=input_)

[n8sty]

What Python does your path default to? ie: Run python --version. Your problem might be happening because the Python available when running python is version 2 while your pyproject.toml specifies version 3.x.

[nickperkins]

That is exactly the issue.

With poetry now having functionality that creates a virtualenv that is compatible with the project (python3) it would also make sense to use that version of python to install the dependences.

[finswimmer]

Hello @nickperkins ,

are you able to reproduce your problem, when you remove the created venv? I guess something went wrong, when your venv was created. poetry should use pip of the venv and not of the system.

Luckily I was able to reproduce your problem using pyvenv to switch the python version for the current shell to python2.7. For any reason it throws an error like here leaving behind a damaged venv.

After a restart of my system and removing this venv everything works as expected. So I guess some update of my system I've made in the meantime was guilty.

fin swimmer

[DmytroLitvinov]

Had the same problem with python2 by default as a global version for python.

[rvanlaar]

@DmytroLitvinov Did you solve this problem and what was your solution?

I have Poetry complaining that pip wasn't installed. Where as pip3 was installed.
After installing pip for python2 it wouldn't install the correct packages.

Another system that is almost the same doesn't have this problem.

Edit: Solved this by:

• installing pip3 as the user
  setting PATH to ~/.local/bin/:$PATH

[finswimmer]

It looks like, that the problem arises if for the specified python interpreter no pip is installed. If this is the case, in the created venv no pip is included, resulting in executing whatever is the global pip.

So make sure you have the correct pip installed for your python version.

[DmytroLitvinov]

@rvanlaar try to use the same version as for poetry with pyenv locally.

[rvanlaar]

This installation has one globally installed python2 (2.7.17) and one globally installed python3 (3.6.9). Both pip2 and pip3 are installed. All of this via packages on Ubuntu 18.04.
The pip executable links to the python2 version of pip.

Poetry worked after:

• removing the poetry created venv
• installing pip3 as the user, which installed it the executable as pip
• configuring the user installed pip to come first in $PATH

@finswimmer

It looks like, that the problem arises if for the specified python interpreter no pip is installed. If this is the case, in the created venv no pip is included, resulting in executing whatever is the global pip.

I'm not sure what to make of this comment. Pip is installed for the specified python interpreter.

@DmytroLitvinov

@rvanlaar try to use the same version as for poetry with pyenv locally.

Pyenv wasn't involved at all.

[drcongo]

Bringing in some extra info from #1817 while trying to get Poetry 1.0 working. My containers end up with globally installed Python 2 and 3, and pip2 and pip3 with a separate executable at /usr/bin/pip which seems to just be the same as pip2. If I rm the pip executable and replace it with a symlink to pip3, and the same with python -> python3 then I'm actually able to install things, but I'm still getting some very odd behaviour.

For instance, adding a new package tells me that it's updating every package that's already installed, but it's updating them from and to the same version...

$ poetry add coverage
Using version ^5.0.1 for coverage

Updating dependencies
Resolving dependencies... (24.8s)

Writing lock file


Package operations: 1 install, 67 updates, 0 removals

  - Updating more-itertools (8.0.2 /usr/lib/python3.6/site-packages -> 8.0.2)
  [...]
  - Updating redis (3.3.11 /usr/lib/python3.6/site-packages -> 3.3.11)
  - Installing coverage (5.0.1)
  - Updating ipdb (0.12.3 /usr/lib/python3.6/site-packages -> 0.12.3)
  - Updating psycopg2 (2.8.4 /usr/lib/python3.6/site-packages -> 2.8.4)
  - Updating pytest-sugar (0.9.2 /usr/lib/python3.6/site-packages -> 0.9.2)

This is when running in a docker container without a virtualenv, if I do it with a virtualenv it tells me it's installing every single one of them every time I add a package.

[drcongo]

Also, running poetry install against the pyproject.toml tells me every single installed package is getting updated from and to the same version, and setting config options doesn't seem to do anything.

$ poetry config --list
cache-dir = "/root/.cache/pypoetry"
virtualenvs.create = false
virtualenvs.in-project = true
virtualenvs.path = "{cache-dir}/virtualenvs"  # /usr/venvs

$ poetry config virtualenvs.create true

$ poetry config --list
cache-dir = "/root/.cache/pypoetry"
virtualenvs.create = false
virtualenvs.in-project = true
virtualenvs.path = "{cache-dir}/virtualenvs"  # /usr/venvs

$ poetry config virtualenvs.create true --local

$ poetry config --list
cache-dir = "/root/.cache/pypoetry"
virtualenvs.create = false
virtualenvs.in-project = true
virtualenvs.path = "{cache-dir}/virtualenvs"  # /usr/venvs

[finswimmer]

Hello @ all,

i still believe something is wrong with your venv's. Can you have a look if the python and pip binarys are available within the venv folders? If not then this issue is similar to #286 and should be fixed with #1797. But the question remains, why the venv folders are not sane.

[rvanlaar]

@finswimmer I can't answer your question. My broken venvs are gone.
I'll report back when I encounter this issue again.

[finswimmer]

Hello @rvanlaar ,

because you didn't come back until now, I guess this issue is solved.

Feel free to leave a comment if you disagree.

fin swimmer

[rvanlaar]

@finswimmer What a nice way of closing this issue. Really makes me feel appreciated.

On topic: Poetry is working well for me and I haven't ran into these problems again. So closed it is.