Poetry: Unable to install using python 3 on clean Ubuntu 18.04 image

[jonapich]

I am trying to install poetry as early as possible in a clean docker run -it ubuntu:18.04 interactive session:

apt update
apt install python3 curl
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3

at this point, poetry installed but won't run because there's no "python" on the system; only python3. so I made a symlink and tried again:

poetry --version

Traceback (most recent call last):
  File "/root/.poetry/lib/poetry/_vendor/py3.6/keyring/backend.py", line 203, in _load_plugins
    init_func = ep.load()
  File "/root/.poetry/lib/poetry/_vendor/py3.6/importlib_metadata/__init__.py", line 92, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/keyrings/alt/Windows.py", line 9, in <module>
    from . import file_base
  File "/usr/lib/python3/dist-packages/keyrings/alt/file_base.py", line 13, in <module>
    from keyring.util.escape import escape as escape_for_ini
ModuleNotFoundError: No module named 'keyring.util.escape'

I retried the steps using apt install python3-pip followed by python3 -m pip install poetry instead of the bootstrap script, but I get the same error when trying to run poetry.

What am I missing exactly?

[jonapich]

I tried the same steps using python2 instead of python3. I get a different outcome:

root@af0c0fe8f81c:~/.poetry/bin# ./poetry --version
/root/.poetry/lib/poetry/_vendor/py2.7/subprocess32.py:149: RuntimeWarning: The _posixsubprocess module is not being used. Child process reliability may suffer if your program uses threads.
  "program uses threads.", RuntimeWarning)
Poetry version 1.0.0
root@af0c0fe8f81c:~/.poetry/bin#

[hutchisr]

Get the same error when installing poetry using pip3 as well. Doesn't seem to be a way to get 1.0.0 to use python3 at all, at least on ubuntu.

[finswimmer]

Hello,

a pip install --upgrade keyrings.alt should fix it.

fin swimmer

[jonapich]

I think I may have added a misstep in my original post... I tried my first scenario again and get a different result:

root@7ebac430ec14:~# poetry
  File "/root/.poetry/lib/poetry/console/commands/shell.py", line 3, in <module>
    from distutils.util import strtobool
ModuleNotFoundError: No module named 'distutils.util'

So I went ahead and apt install python3-pip. It's a 110mb download with all its dependencies. Only then, I can reproduce the issue from my original post. (EDIT: you can also get python3-distutils which is a much smaller package; in both cases, we end up with the keyring callstack)

I tried updating keyrings.alt as suggested. It seems to fix it, and then it breaks again later on:

Successfully installed keyrings.alt-3.4.0
root@7ebac430ec14:~# poetry
Poetry version 1.0.0

USAGE

[UnicodeEncodeError]
'ascii' codec can't encode character '\xa0' in position 30: ordinal not in range(128)

[aizelauna]

Hi,

On my side, I ended up by installing _poetry_ with pipx.

It runs only with python3 but that's the most stable install method I found on Ubuntu 18.04. I do not have any issue since several weeks. It solves also the issue #1257.

[finswimmer]

Hello,

I try to reproduce the problems mentioned here, by installing a fresh ubuntu 18.04 in a virtual machine. After doing this (installing the "minimal" variant), I ended up with a system where a python3 exists but no python. At the moment, poetry expects that there is a python within the path. It doesn't matter if it links to a python2 or python3. So I install python2 by sudo apt-get install python. After that it looks like this

$ which python
/usr/bin/python

$ python --version
Python 2.7.17

$ which python2
/usr/bin/python2

$ python2 --version
Python 2.7.17

$ which python3
/usr/bin/python3

$ python3 --version
Python 3.6.9

For installing poetry, curl is needed:

sudo apt-get install curl

Then:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python

Now let's create two poetry projects. One that should use python2 and one python3.

mkdir project_python2
cd project_python2
poetry init --no-interaction

The pyproject.toml looks like this:

[tool.poetry]
name = "project_python2"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]

[tool.poetry.dependencies]
python = "^2.7"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Now install the project:

$ poetry install

And you're done. Whenever running a poetry command you will receive a warning like this:

/home/finswimmer/.poetry/lib/poetry/_vendor/py2.7/subprocess32.py:149: RuntimeWarning: The _posixsubprocess module is not being used. Child process reliability may suffer if your program uses threads.  "program uses threads.", RuntimeWarning)

It's a warning and AFAIK you can ignore it.

Now let's see what happens for a project uses python3. Copy the above pyproject.toml to a new folder called project_python3. Change the name in this file to match the folder name and change
python = "^2.7" to python = "^3.6".

Run poetry install and you may receive an error message that you need to install python3-venv. Ok, let's do it:

$ sudo apt-get install python3-venv

Unfortunately poetry left a broken venv behind, we had to remove before trying again.

$ poetry env remove python3

and

$ poetry install
$ poetry run python --version
Python 3.6.9

To summarize:

• make sure there is a python
• it doesn't matter if python links to python2 or python3
• in addition to python there should be python2 and python3 in your path, if you like to use different python3 versions in different project. Because poetry tries the one or the other if the version of python doesn't match the project's python version
• You may need to install python3-venv. But poetry will complain about it if needed.

fin swimmer

[jonapich]

@finswimmer thanks for double-checking; I already knew that python 2 worked to install it. The documentation is clear that it doesn't matter which version of python is used to install poetry, so the scenario outlined in my original post (with python 3) is clearly an issue.

Using python 2 is not acceptable in my case; it's simply not available on our production images and our python executable launches py3. Poetry absolutely have to be able to work in a python2-less environment.

[jonapich]

Tried some other scenarios this morning:

works:

docker run -it python:3 bash
python -m pip install poetry

works:

docker run -it python:3 bash
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python

breaks with keyring bug:

docker run -it ubuntu:18.04 bash
apt update && apt install python3-pip
python3 -m pip install poetry

Why does the bootstrap script/pip works on the python3 image (which does NOT have a python 2 installed; python points to py3) and not the ubuntu one? :thinking:

[jonapich]

Breaks with [UnicodeEncodeError]:

docker run -it ubuntu:18.04 bash
apt update && apt install python3-pip python3-venv
python3 -m pip install pipx
pipx install poetry

[jonapich]

I wrote a different issue for the unicode bug: #1849

[jonapich]

(can someone tag this issue as a bug?)

[pcgeek86]

Same issue here. Just reproduced it on a couple of brand new Ubuntu VMs, using Multipass.

Reproduction Steps

# 1. Create a new Multipass Ubuntu VM
multipass shell

# 2. Install Python3
sudo apt-get install python3-pip

# 3. Install poetry
sudo -H pip3 install poetry --upgrade

# 4. Run poetry
poetry

Result

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/keyring/backend.py", line 203, in _load_plugins
    init_func = ep.load()
  File "/usr/local/lib/python3.6/dist-packages/importlib_metadata/__init__.py", line 92, in load
    module = import_module(match.group('module'))
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/lib/python3/dist-packages/keyrings/alt/Windows.py", line 9, in <module>
    from . import file_base
  File "/usr/lib/python3/dist-packages/keyrings/alt/file_base.py", line 13, in <module>
    from keyring.util.escape import escape as escape_for_ini
ModuleNotFoundError: No module named 'keyring.util.escape'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/poetry", line 7, in <module>
    from poetry.console import main
  File "/usr/local/lib/python3.6/dist-packages/poetry/console/__init__.py", line 1, in <module>
    from .application import Application
  File "/usr/local/lib/python3.6/dist-packages/poetry/console/application.py", line 5, in <module>
    from .commands.about import AboutCommand
  File "/usr/local/lib/python3.6/dist-packages/poetry/console/commands/__init__.py", line 4, in <module>
    from .check import CheckCommand
  File "/usr/local/lib/python3.6/dist-packages/poetry/console/commands/check.py", line 1, in <module>
    from poetry.factory import Factory
  File "/usr/local/lib/python3.6/dist-packages/poetry/factory.py", line 17, in <module>
    from .packages.dependency import Dependency
  File "/usr/local/lib/python3.6/dist-packages/poetry/packages/__init__.py", line 7, in <module>
    from .dependency import Dependency
  File "/usr/local/lib/python3.6/dist-packages/poetry/packages/dependency.py", line 10, in <module>
    from poetry.utils.helpers import canonicalize_name
  File "/usr/local/lib/python3.6/dist-packages/poetry/utils/helpers.py", line 11, in <module>
    from keyring import delete_password
  File "/usr/local/lib/python3.6/dist-packages/keyring/__init__.py", line 1, in <module>
    from .core import (
  File "/usr/local/lib/python3.6/dist-packages/keyring/core.py", line 192, in <module>
    init_backend()
  File "/usr/local/lib/python3.6/dist-packages/keyring/core.py", line 96, in init_backend
    filter(limit, backend.get_all_keyring()),
  File "/usr/local/lib/python3.6/dist-packages/keyring/util/__init__.py", line 22, in wrapper
    func.always_returns = func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/keyring/backend.py", line 216, in get_all_keyring
    _load_plugins()
  File "/usr/local/lib/python3.6/dist-packages/keyring/backend.py", line 207, in _load_plugins
    log.exception("Error initializing plugin %s." % ep)
TypeError: not all arguments converted during string formatting

[mbello]

Same issue here. Just reproduced it on a couple of brand new Ubuntu VMs, using Multipass.

Reproduction Steps

# 1. Create a new Multipass Ubuntu VM
multipass shell

# 2. Install Python3
sudo apt-get install python3-pip

# 3. Install poetry
sudo -H pip3 install poetry --upgrade

# 4. Run poetry
poetry

Will work if first you run
pip3 install --upgrade keyrings.alt

[pcgeek86]

I'm not sure if this is the correct long-term fix for poetry, but do we need to add keyrings.alt to the list of poetry dependencies in this file? I can confirm that installing this package causes poetry to start working.

https://github.com/python-poetry/poetry/blob/master/pyproject.toml#L52

[ShamrockLee]

This issue also happens to Ubuntu 20.04

[abn]

@jonapich this works for me at the moment; can you confirm please?

docker run --rm -it ubuntu:18.04 bash -c "apt -y update; apt install -y python3 python3-pip curl; curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3; source \$HOME/.poetry/env; export PYTHONIOENCODING=utf-8; poetry --version; poetry --help"

Note that that the ascii encoding error is an issuew ith the ubuntu docker image locale. In the above command I make use of export PYTHONIOENCODING=utf-8 to make that work. See https://github.com/python-poetry/poetry/issues/1719#issuecomment-565391165.

And the distutils requirement is expected to be available in a python install. I am not sure how Ubuntu is building their python package to comment on why it does not exist there.

@pcgeek86 regarding keyring.alt issue, see https://github.com/python-poetry/poetry/issues/1719#issuecomment-565846793. The package is not required by poetry and hence should not be indicated as a dependency. This is a distro issue unfortunately and there is not clean fix that can be done from the poetry side for this one.

See also: #1719