Poetry: Poetry using the wrong Python version (not related to pyenv)

[zmitchell]

• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have searched the documentation and believe that my question is not covered.

Question

I'm having an issue adding a package to a project because poetry is picking up on the wrong Python version. I'm pretty confident that something has gone awry in my system, but I'm doing my best to salvage the situation and not have to go through the whole process of installing Raspbian from scratch on this Raspberry Pi. I installed poetry using the recommended method (curl ...). Can you point me in the right direction? As mentioned in the title, I'm not using pyenv. All of the relevant issues I found were related to pyenv, so I'm not really sure how to get poetry to use the correct Python.

Here are some terminal outputs to help you diagnose the issue:

pi@widget:temp_logger$ python --version
Python 3.5.3
pi@widget:temp_logger!$ sudo python --version
Python 2.7.13
pi@widget:temp_logger$ poetry --version
Poetry 0.12.9
pi@widget:temp_logger$ poetry debug:info

Poetry
======

 * Version: 0.12.9
 * Python:  2.7.13


Virtualenv
==========

 * Python:         2.7.13
 * Implementation: CPython
 * Path:           NA


System
======

 * Platform: linux2
 * OS:       posix
 * Python:   /usr


pi@widget:temp_logger$ poetry add adafruit-blinka

[RuntimeError]
The current Python version (2.7.13) is not supported by the project (^3.5)
Please activate a compatible Python version.

add [-D|--dev] [--git GIT] [--path PATH] [-E|--extras EXTRAS] [--optional] [--python PYTHON] [--platform PLATFORM] [--allow-prereleases] [--dry-run] [--] <name> (<name>)...

[sdispater]

As you can see in debug:info, Poetry picks up Python 2.7.13 and not Python 3.5.3. What is the output of which python? Does it point to your 2.7.13 Python or 3.5.3?

[zmitchell]

Ah, here we go. It turns out python was aliased to python3, and doing which python uncovered that. Now python --version returns 2.7.13, and which python returns /usr/bin/python. So how do I get poetry to use Python 3 instead?

[buriy]

@sdispater the issue is even worse.

Poetry starting with 0.12.5 stopped working with the python version from the virtualenv, only checks system version of Python.
Version 0.12.4 is the last version that worked propertly.

After this command $ sudo pip3 install poetry it shows:

$ poetry update
[RuntimeError]                                                
The current Python version (3.6.7) is not supported by the project (~2.7)  
Please activate a compatible Python version.                               

$ poetry debug:info

Poetry
======

 * Version: 0.12.9
 * Python:  3.6.7


Virtualenv
==========

 * Python:         2.7.15
 * Implementation: CPython
 * Path:           /Projects/myproject/.venv
 * Valid:          True


System
======

 * Platform: linux
 * OS:       posix
 * Python:   /usr

If you would install it with sudo pip2 install poetry, it will work with python 2.7 projects, but won't work with python 3.6 projects.
I don't use "source .venv/bin/activate" and I shouldn't be forced to.

[sdispater]

@buriy Commit 94304e9 should fix your issue.

It will land in the next bugfix release.

[buriy]

@sdispater Thanks, then these are different issues with the one from topic starter.
Sorry for hijacking thread.

[buriy]

@zmitchell Then, If I got it right, try reinstalling poetry. If installed with python2, it's bound to using python2 unless you use pyenv or virtualenv.

[zmitchell]

I reinstalled with Python 3 using curl ... | python3 and it still only picks up 2.7.

[jtbeach]

@zmitchell In 0.12.x, poetry is not bound to the interpreter version you curl the installer script through during installation -- so it doesn't matter which one you install it with with.

If python points to python2 and your project needs python3, you have 2 choices:

1. (preferable) Use pyenv and run pyenv local 3.7.0 or whatever your project wants in the root folder of your project. If you have your pyenv shims setup correctly in your shell startup files, python will now execute Python 3.7.0 whenever python runs from that folder or its subfolders, regardless of what python points to globally.

2. Manually create a virtualenv with Python 3 and activate it prior to running poetry commands. You can use direnv or similar to make that a little more automated, but I still prefer using pyenv -- once you get used to it, it's great

[buriy]

@jtbeach

1. Manually create a virtualenv with Python 3 and activate it prior to running poetry commands. You can use direnv or similar to make that a little more automated, but I still prefer using pyenv -- once you get used to it, it's great

Plese note that this option for the local .venv folder is broken in version 0.12.5 and will be fixed in version 0.12.10, which is not yet published (I rechecked it's not fixed in 0.12.9).

[sdispater]

@zmitchell For now, your best bet is to use pyenv (which is the easiest way to manager different Python versions if you are not on Windows) until #621 is implemented (work in progress here https://github.com/sdispater/poetry/tree/env-command)

[sdispater]

@buriy Release 0.12.10 is out! 

[buriy]

@sdispater thanks a lot!
Option 2 mentioned by @jtbeach works again.

[asodeur]

Not sure if this is the original issue or a new Windows related issue (seeing this on 0.12.10 and 1.0.0a1).

SystemEnv.run("python", ...) will always use the primary system interpreter on Windows even if poetry is running on another one not installed in a virtual env. This is because SystemEnv._bin("python") is looking for python.exe in SystemEnv._bin_dir which on Windows is <Python root>/Scripts, however the interpreter is there for virtual envs only. for standard Python installations the interpreter is <Python root>/python.exe.

Running poetry from inside a virtual env, ie option 2 above works fine.

[airstandley]

I've also been running into issues trying to get poetry installing to the correct version of python when not using a virtual environment (poetry config settings.virtualenvs.create false).

After doing some digging it looks like SystemEnv is still determining the binary to call by appending sys.base_prefix/bin to the executable name given. This approach will not work on linux systems that have versions of both python2 and python3 installed.
It seems that a system python environment should be using the system python and simply calling the binary without a specific path. At the moment there seems to be no way to get poetry to install to a linux system python3 installation without changing the /usr/bin/python symlink prior to running poetry install. This is far from ideal. At least if just python was used a shim could be inserted on the path which would allow poetry to install to the correct python version without risking issues with system libraries that call /usr/bin/python.

Ideally it would be nice to just use the version of python poetry was executed with when installing to the system environment.

[Peque]

Using Poetry 1.0 (prerelease) seems to fix this issue:

poetry self:update --preview
poetry env use python3.7

See https://github.com/sdispater/poetry/pull/731 for more information.

[maggyero]

@Peque Well the new poetry env command provides an _explicit_ way to fix the issue (which is great). But the _implicit_ way is still broken. Poetry should implicitly create a built-in virtual environment with the Python version specified by the tool.poetry.dependencies.python property of the pyproject.toml file, not the Python version resolved by the #!/usr/bin/env python shebang line of the ~/.poetry/bin/poetry script.

[sdispater]

This has been improved in the latest beta release (see #1477 for more information).

[tatianafrank]

The only thing that works for me (until i can use the new env feature) is running pyenv shell <version>. Unfortunately pyenv local and pyenv global dont work and my virtual environment is not getting picked up otherwise. Obviously this is not a good solution but a workaround for now..

[AlJohri]

~Upgrading to the latest beta (1.0.0b8) fixed the issue for me:~

The currently activated Python version 3.7.5 is not supported by the project (^3.8).
Trying to find and use a compatible version.
Using python3 (3.8.0)
Creating virtualenv al-research-project in /Users/aljohri/Development/al-research-project/.venv

p.s. hi @tatianafrank!

EDIT: Actually, no it didn't fix it. Despite the message that says it will use 3.8.0, the virtualenv uses poetry's python version:

$ /Users/aljohri/Development/al-research-project/.venv/bin/python --version
3.7.5

Here's some debug information:

As you can see, I'm using asdf (which is basically equivalent to pyenv but allows for multiple languages).

Additionally, I have poetry installed via pipx (a common way to install python CLI tools in their own isolated virtualenvs):

$ pipx install poetry --pip-args=--pre
  installed package poetry 1.0.0b8, Python 3.7.5
  These apps are now globally available
    - poetry
done! ✨ 🌟 ✨
$ which poetry
/Users/aljohri/.local/bin/poetry

[ekhaydarov]

None of the above works if youre trying to run poetry from crontab

[yifeikong]

Using curl -sSL ... to install is still broken on macOS catalina with homebrewed python3. However, using pip3 install poetry works for me.

[JulianFerry]

I think it would be nice to be able to specify poetry config python-version 3.7 so that poetry generally defaults to that version of python.

For example poetry new should default to using that version when creating a new project. Currently I have to remember to update the python version from 2.7 to 3.7. In the cases where I forget that leads to some confusion when I call poetry add but can't access that package in my python interpreter.

Currently when I call poetry new, if I then call poetry env use python3.7 I get:

[NoCompatiblePythonVersionFound]
The specified Python version (3.7.7) is not supported by the project (^2.7).
Please choose a compatible version or loosen the python constraint specified in the pyproject.toml file.`

If I edit the pyproject.toml file to python = "^3.7" instead of 2.7, when I call poetry add I see the following warning message:

The currently activated Python version 2.7.16 is not supported by the project (^3.7).
Trying to find and use a compatible version.
Using python3 (3.7.7)

Which is slightly confusing since no version of python has been "activated" at this point, and I have aliased python to python3, so it is only the poetry env use configuration which is incorrect.

[GabrielBB]

Using curl -sSL ... to install is still broken on macOS catalina with homebrewed python3. However, using pip3 install poetry works for me.

This did it for me! Don't forget to remove the old poetry from your path

[johnthagen]

I'd like to echo @maggyero's comment: https://github.com/python-poetry/poetry/issues/655#issuecomment-532608560

As a first time new user, installing poetry using the recommended curl method, it was very strange to me that by default poetry install created a Python 2.7 virtual environment for a project marked as Python 3 in pyproject.toml. Especially since Python 2 is end-of-life, there is extra importance in my mind to keep this from happening on new users.

[dyerdave-cvs]

Also not using pyenv. pip3 install poetry worked for me, whereas all the other suggestions did not. On Amazon Linux 2, which still runs 2.7 by default.

[mtsolmn]

If I edit the _pyproject.toml_ file to python = "^3.7" instead of 2.7, when I call poetry add I see the following warning message:

The currently activated Python version 2.7.16 is not supported by the project (^3.7).
Trying to find and use a compatible version.
Using python3 (3.7.7)

Which is slightly confusing since no version of python has been "activated" at this point, and I have aliased python to python3, so it is only the poetry env use configuration which is incorrect.

@JulianFerry If you have already ran poetry install, then a poetry.lock file have been created and is the reason for this issue. I do not know of any argument for poetry add to ignore poetry.lock, so you may need to delete poetry.lock after editing pyproject.toml.

Poetry should implicitly create a built-in virtual environment with the Python version specified by the tool.poetry.dependencies.python property of the pyproject.toml file, not the Python version resolved by the #!/usr/bin/env python shebang line of the ~/.poetry/bin/poetry script.

To give a temporary solution to @maggyero and @johnthagen, yes I agree this is really annoying. For me the best solution on Mac OS has been to manually edit the shebang line of ~/.poetry/bin/poetry to #!/usr/bin/env python3 and I have python3 as a symlink to my python3.8 version from homebrew. Now poetry uses my homebrew version of python to run and, equally important, now sets the python dependency of any new project as python = "^3.8".

❯ poetry new test-demo
Created package test_demo in test-demo
❯ cd test-demo
❯ poetry env info

Virtualenv
Python:         3.8.5
Implementation: CPython
Path:           NA

System
Platform: darwin
OS:       posix
Python:   /usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8

This is a relatively elegant, although inconvenient solution to the problem. But you can set it once and forget it, and can modify the shebang line for whatever version of python you want. I think that poetry should have a config option to set the python PATH for the executable it runs and a separate config option for the default dependency version of python set in pyproject.toml for any new project created. Even better would be more argument flags for poetry new to set these things per project at creation time. But this is what we have for now. Hope this helps.

[benoit-fuentes]

Hi all
@mtsolmn Thanks for the #!/usr/bin/env python3 hack, this is the only thing that works for me (by default any project created with the new command would be only python2.7 compatible, and the poetry env use python3 command would end up with a [NoCompatiblePythonVersionFound] error.

An idea to prevent that would be to be able to add a tag that specifies python version when creating a new project. Something like

poetry new mypck --python_version 3.7+

Or at least why it does not update the pyproject.toml file when using poetry env use python3

[martynasoxylabs]

I solve this problem on my Mac by replacing following line

#!/usr/bin/env python

to

```

!/usr/bin/env python3.8

`` in/home//.poetry/bin/poetry` file. Then it picks python3.8 automatically and there are no annoying warnings anymore.