Poetry: FileNotFound error when installing packages when console script same name as package for Python 2.7

Created on 16 Oct 2020 · 14Comments · Source: python-poetry/poetry

[x] I am on the latest Poetry version.
[x] I have searched the issues of this repo and believe that this is not a duplicate.
[x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: Ubuntu 20.04
Poetry version: 1.1.3
Link of a Gist with the contents of your pyproject.toml file:

Issue

If I have a console script with the same name as the package, I get a FileNotFound error when I do poetry install in a python 2.7 virtualenv:

$ poetry init mytest
# Put the pyproject.toml from my gist in the root directory of the package
# Put the main.py from my gist in the mytest directory
$ poetry update
$ poetry env use 2.7
$ poetry install -vvv
Using virtualenv: /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 0 installs, 18 updates, 0 removals, 1 skipped

  • Removing colorama (0.4.4): Pending...
  • Removing colorama (0.4.4): Skipped for the following reason: Not currently installed
  • Updating contextlib2 (0.6.0.post1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.6.0.post1): Pending...
  • Updating contextlib2 (0.6.0.post1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.6.0.post1): Installing...
  • Updating contextlib2 (0.6.0.post1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.6.0.post1)
  • Updating scandir (1.10.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.10.0): Pending...
  • Updating scandir (1.10.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.10.0): Installing...
  • Updating scandir (1.10.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.10.0)
  • Updating six (1.15.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.15.0): Pending...
  • Updating six (1.15.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.15.0): Installing...
  • Updating six (1.15.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.15.0)
  • Updating configparser (4.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.0.2): Pending...
  • Updating configparser (4.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.0.2): Installing...
  • Updating configparser (4.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.0.2)
  • Updating pathlib2 (2.3.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.3.5): Pending...
  • Updating pathlib2 (2.3.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.3.5): Installing...
  • Updating pathlib2 (2.3.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.3.5)
  • Updating zipp (1.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.2.0): Pending...
  • Updating zipp (1.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.2.0): Installing...
  • Updating zipp (1.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.2.0)
  • Updating backports.functools-lru-cache (1.6.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.6.1): Pending...
  • Updating backports.functools-lru-cache (1.6.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.6.1): Installing...
  • Updating backports.functools-lru-cache (1.6.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.6.1)
  • Updating importlib-metadata (2.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.0.0): Pending...
  • Updating importlib-metadata (2.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.0.0): Installing...
  • Updating importlib-metadata (2.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.0.0)
  • Updating pyparsing (2.4.7 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.4.7): Pending...
  • Updating pyparsing (2.4.7 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.4.7): Installing...
  • Updating pyparsing (2.4.7 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.4.7)
  • Updating atomicwrites (1.4.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.4.0): Pending...
  • Updating atomicwrites (1.4.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.4.0): Installing...
  • Updating attrs (20.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.2.0): Pending...
  • Updating attrs (20.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.2.0): Installing...
  • Updating atomicwrites (1.4.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.4.0)
  • Updating attrs (20.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.2.0): Pending...
  • Updating attrs (20.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.2.0): Installing...
  • Updating funcsigs (1.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.0.2): Pending...
  • Updating funcsigs (1.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.0.2): Installing...
  • Updating funcsigs (1.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.0.2)
  • Updating attrs (20.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.2.0)
  • Updating funcsigs (1.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.0.2): Pending...
  • Updating funcsigs (1.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.0.2): Installing...
  • Updating funcsigs (1.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.0.2)
  • Updating more-itertools (5.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 5.0.0): Pending...
  • Updating more-itertools (5.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 5.0.0): Installing...
  • Updating more-itertools (5.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 5.0.0)
  • Updating packaging (20.4 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.4): Pending...
  • Updating packaging (20.4 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.4): Installing...
  • Updating packaging (20.4 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.4)
  • Updating pluggy (0.13.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.13.1): Pending...
  • Updating pluggy (0.13.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.13.1): Installing...
  • Updating pluggy (0.13.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.13.1)
  • Updating py (1.9.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.9.0): Pending...
  • Updating py (1.9.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.9.0): Installing...
  • Updating py (1.9.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.9.0)
  • Updating wcwidth (0.2.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.2.5): Pending...
  • Updating wcwidth (0.2.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.2.5): Installing...
  • Updating wcwidth (0.2.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.2.5)
  • Updating pytest (4.6.11 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.6.11): Pending...
  • Updating pytest (4.6.11 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.6.11): Installing...
  • Updating pytest (4.6.11 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.6.11)

Installing the current project: mytest (0.1.0)
  - Building package mytest in editable mode
  - Adding mytest.pth to /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/local/lib/python2.7/dist-packages for /home/developer/projects/poetry/mytest
  - Adding the mytest script to /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/local/bin

  Stack trace:

  8  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│ 
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  7  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  6  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│ 
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│ 
      173│     def __repr__(self):  # type: () -> str

  5  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│ 
    →  92│         return self.handle()
       93│ 
       94│     def handle(self):  # type: () -> Optional[int]

  4  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/poetry/console/commands/install.py:105 in handle
      103│             return 0
      104│ 
    → 105│         builder.build()
      106│ 
      107│         if self._io.supports_ansi() and not self.io.is_debug():

  3  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/poetry/masonry/builders/editable.py:57 in build
       55│         added_files = []
       56│         added_files += self._add_pth()
    →  57│         added_files += self._add_scripts()
       58│         self._add_dist_info(added_files)
       59│ 

  2  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/poetry/masonry/builders/editable.py:150 in _add_scripts
      148│                 )
      149│             )
    → 150│             with script_file.open("w", encoding="utf-8") as f:
      151│                 f.write(
      152│                     decode(

  1  /usr/lib/python3.7/pathlib.py:1208 in open
      1206│             self._raise_closed()
      1207│         return io.open(self, mode, buffering, encoding, errors, newline,
    → 1208│                        opener=self._opener)
      1209│ 
      1210│     def read_bytes(self):

  FileNotFoundError

  [Errno 2] No such file or directory: '/home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/local/bin/mytest'

  at /usr/lib/python3.7/pathlib.py:1063 in _opener
      1059│         raise ValueError("I/O operation on closed path")
      1060│ 
      1061│     def _opener(self, name, flags, mode=0o666):
      1062│         # A stub for the opener argument to built-in open()
    → 1063│         return self._accessor.open(self, flags, mode)
      1064│ 
      1065│     def _raw_open(self, flags, mode=0o777):
      1066│         
      1067│         Open the file pointed by this path and return a file descriptor,

This doesn't happen in python 3.7:

$ poetry env use 3.7
$ poetry install
Installing dependencies from lock file

Package operations: 12 installs, 0 updates, 0 removals

  • Installing zipp (1.2.0)
  • Installing importlib-metadata (2.0.0)
  • Installing pyparsing (2.4.7)
  • Installing six (1.15.0)
  • Installing atomicwrites (1.4.0)
  • Installing attrs (20.2.0)
  • Installing more-itertools (5.0.0)
  • Installing packaging (20.4)
  • Installing pluggy (0.13.1)
  • Installing py (1.9.0)
  • Installing wcwidth (0.2.5)
  • Installing pytest (4.6.11)

Installing the current project: mytest (0.1.0)

Bug Triage

Source

rzuckerm

All 14 comments

This is currently blocking us from using the latest version of poetry. We still have a lot of python 2.7 in our codebases ☹️

rzuckerm on 23 Oct 2020

@rzuckerm I was unable to reproduce it. Am I missing something?

podman run --rm -i --entrypoint bash python:2.7 <<EOF
set -xe
python -m pip install -q poetry
git clone https://gist.github.com/rzuckerm/1db0c59c367525d70d618129329d6a50.git 3218
pushd 3218
sed -i /pytest/d pyproject.toml  # remove pytest since we do not need it
install -d mytest
touch mytest/__init__.py
mv main.py mytest/.
poetry install
poetry run mytest
poetry run which mytest
EOF

+ python -m pip install -q poetry
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
ERROR: poetry-core 1.0.0 has requirement importlib-metadata<2.0.0,>=1.7.0; python_version >= "2.7" and python_version < "2.8" or python_version >= "3.5" and python_version < "3.8", but you'll have importlib-metadata 1.6.0 which is incompatible.
WARNING: You are using pip version 20.0.2; however, version 20.2.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
+ git clone https://gist.github.com/rzuckerm/1db0c59c367525d70d618129329d6a50.git 3218
Cloning into '3218'...
+ pushd 3218
/3218 /
+ sed -i /pytest/d pyproject.toml
+ install -d mytest
+ touch mytest/__init__.py
+ mv main.py mytest/.
+ poetry install

Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
You should consider updating your Python version to a supported one.

Note that you will still be able to manage Python 2.7 projects by using the env command.
See https://python-poetry.org/docs/managing-environments/ for more information.

Creating virtualenv mytest-fMhv_1Ox-py2.7 in /root/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies...

Writing lock file

Installing the current project: mytest (0.1.0)
+ poetry run mytest

Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
You should consider updating your Python version to a supported one.

Note that you will still be able to manage Python 2.7 projects by using the env command.
See https://python-poetry.org/docs/managing-environments/ for more information.

+ poetry run which mytest

Python 2.7 will no longer be supported in the next feature release of Poetry (1.2).
You should consider updating your Python version to a supported one.

Note that you will still be able to manage Python 2.7 projects by using the env command.
See https://python-poetry.org/docs/managing-environments/ for more information.

/root/.cache/pypoetry/virtualenvs/mytest-fMhv_1Ox-py2.7/bin/mytest

abn on 24 Oct 2020

I've installed the latest poetry in a python 3.7 virtualenv. I run all the poetry commands from this virtualenv. I can reproduce this issue on demand. I use the following to install the packages into a python 2.7 virtualenv:

poetry env remove 2.7
poetry env use 2.7
poetry install

The original issue was for poetry 1.1.3, but with poetry 1.1.4, I can reproduce this issue without the console script. In poetry 1.1.3, the issue would not happen if I commented out the console script.

This is what I see now:

$ poetry install -vvv
Using virtualenv: /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 0 installs, 18 updates, 0 removals, 1 skipped

  • Removing colorama (0.4.4): Pending...
  • Removing colorama (0.4.4): Skipped for the following reason: Not currently installed
  • Updating contextlib2 (0.6.0.post1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.6.0.post1): Pending...
  • Updating contextlib2 (0.6.0.post1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.6.0.post1): Installing...
  • Updating contextlib2 (0.6.0.post1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.6.0.post1)
  • Updating scandir (1.10.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.10.0): Pending...
  • Updating scandir (1.10.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.10.0): Installing...
  • Updating scandir (1.10.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.10.0)
  • Updating six (1.15.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.15.0): Pending...
  • Updating six (1.15.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.15.0): Installing...
  • Updating six (1.15.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.15.0)
  • Updating configparser (4.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.0.2): Pending...
  • Updating configparser (4.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.0.2): Installing...
  • Updating configparser (4.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.0.2)
  • Updating pathlib2 (2.3.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.3.5): Pending...
  • Updating pathlib2 (2.3.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.3.5): Installing...
  • Updating pathlib2 (2.3.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.3.5)
  • Updating zipp (1.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.2.0): Pending...
  • Updating zipp (1.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.2.0): Installing...
  • Updating zipp (1.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.2.0)
  • Updating backports.functools-lru-cache (1.6.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.6.1): Pending...
  • Updating backports.functools-lru-cache (1.6.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.6.1): Installing...
  • Updating backports.functools-lru-cache (1.6.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.6.1)
  • Updating importlib-metadata (2.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.0.0): Pending...
  • Updating importlib-metadata (2.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.0.0): Installing...
  • Updating importlib-metadata (2.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.0.0)
  • Updating pyparsing (2.4.7 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.4.7): Pending...
  • Updating pyparsing (2.4.7 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.4.7): Installing...
  • Updating pyparsing (2.4.7 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 2.4.7)
  • Updating atomicwrites (1.4.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.4.0): Pending...
  • Updating atomicwrites (1.4.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.4.0): Installing...
  • Updating atomicwrites (1.4.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.4.0)
  • Updating attrs (20.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.2.0): Pending...
  • Updating attrs (20.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.2.0): Installing...
  • Updating attrs (20.2.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.2.0)
  • Updating funcsigs (1.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.0.2): Pending...
  • Updating funcsigs (1.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.0.2): Installing...
  • Updating funcsigs (1.0.2 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.0.2)
  • Updating more-itertools (5.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 5.0.0): Pending...
  • Updating more-itertools (5.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 5.0.0): Installing...
  • Updating more-itertools (5.0.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 5.0.0)
  • Updating packaging (20.4 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.4): Pending...
  • Updating packaging (20.4 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.4): Installing...
  • Updating packaging (20.4 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 20.4)
  • Updating pluggy (0.13.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.13.1): Pending...
  • Updating pluggy (0.13.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.13.1): Installing...
  • Updating pluggy (0.13.1 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.13.1)
  • Updating py (1.9.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.9.0): Pending...
  • Updating py (1.9.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.9.0): Installing...
  • Updating py (1.9.0 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 1.9.0)
  • Updating wcwidth (0.2.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.2.5): Pending...
  • Updating wcwidth (0.2.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.2.5): Installing...
  • Updating wcwidth (0.2.5 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 0.2.5)
  • Updating pytest (4.6.11 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.6.11): Pending...
  • Updating pytest (4.6.11 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.6.11): Installing...
  • Updating pytest (4.6.11 /home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7/site-packages -> 4.6.11)

Installing the current project: mytest (0.1.0)
  - Building package mytest in editable mode
  - Failed to create mytest.pth for /home/developer/projects/poetry/mytest

  Stack trace:

  8  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│ 
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  7  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  6  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│ 
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│ 
      173│     def __repr__(self):  # type: () -> str

  5  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│ 
    →  92│         return self.handle()
       93│ 
       94│     def handle(self):  # type: () -> Optional[int]

  4  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/poetry/console/commands/install.py:105 in handle
      103│             return 0
      104│ 
    → 105│         builder.build()
      106│ 
      107│         if self._io.supports_ansi() and not self.io.is_debug():

  3  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/poetry/masonry/builders/editable.py:59 in build
       57│         added_files += self._add_pth()
       58│         added_files += self._add_scripts()
    →  59│         self._add_dist_info(added_files)
       60│ 
       61│     def _run_build_script(self, build_script):

  2  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/poetry/masonry/builders/editable.py:207 in _add_dist_info
      205│                 shutil.rmtree(str(dist_info))
      206│ 
    → 207│         dist_info = self._env.site_packages.mkdir(dist_info_path)
      208│ 
      209│         self._debug(

  1  ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/poetry/utils/env.py:238 in mkdir
       236│ 
       237│     def mkdir(self, path, *args, **kwargs):  # type: (Path, *Any, **Any) -> Path
    →  238│         return self._path_method_wrapper(path, "mkdir", *args, **kwargs)[0]
       239│ 
       240│     def exists(self, path):  # type: (Path) -> bool

  OSError

  Unable to access any of "/home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/local/lib/python2.7/dist-packages/mytest-0.1.0.dist-info"

  at ~/virtualenvs/poetry_1.1.x/lib/python3.7/site-packages/poetry/utils/env.py:232 in _path_method_wrapper
       228│ 
       229│         if results:
       230│             return results
       231│ 
    →  232│         raise OSError("Unable to access any of {}".format(paths_csv(candidates)))
       233│ 
       234│     def write_text(self, path, *args, **kwargs):  # type: (Path, *Any, **Any) -> Path
       235│         return self._path_method_wrapper(path, "write_text", *args, **kwargs)[0]
       236│

rzuckerm on 24 Oct 2020

I think I figured out what is going on for the python 2.7 virtualenv, this is the sysconfig paths:

$ ~/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/bin/python
Python 2.7.18 (default, Aug  4 2020, 11:16:42) 
[GCC 9.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> import pprint
>>> pprint.pprint(sysconfig.get_paths())
{'data': '/home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/local',
 'include': '/home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/local/include/python2.7',
 'platinclude': '/home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/local/include/python2.7',
 'platlib': '/home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/local/lib/python2.7/dist-packages',
 'platstdlib': '/home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7',
 'purelib': '/home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/local/lib/python2.7/dist-packages',
 'scripts': '/home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/local/bin',
 'stdlib': '/home/developer/.cache/pypoetry/virtualenvs/mytest-qD4dPpKx-py2.7/lib/python2.7'}

When the python 2.7 virtualenv is created, the local directory does not exist. Somewhere along the way, poetry seems to create the local and local/bin directories. The local/bin directory is empty. It seems that poetry is trying to write a file to a directory in local that does not exist.

I managed to work around this problem by changing this line to this:

dist_info = self._env.site_packages.mkdir(dist_info_path, parents=True)

If I make that change, then install works, but I see this:

Installing the current project: mytest (0.1.0)  - Failed to create mytest.pth for /home/developer/projects/poetry/mytest

I can fix that by adding this before this this line:

self._env.site_packages.path.mkdir(parents=True, exist_ok=True)

rzuckerm on 24 Oct 2020

When the python 2.7 virtualenv is created, the local directory does not exist.

I wonder where the local directory comes from in your case :thinking: This is, how it looks here:

Python 2.7.17 (default, Oct  3 2020, 10:45:01) 
[GCC 10.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> import pprint
>>> pprint.pprint(sysconfig.get_paths())
{'data': '/home/finswimmer/.cache/pypoetry/virtualenvs/mytest-Tqz13LJJ-py2.7',
 'include': '/home/finswimmer/.cache/pypoetry/virtualenvs/mytest-Tqz13LJJ-py2.7/include/python2.7',
 'platinclude': '/home/finswimmer/.cache/pypoetry/virtualenvs/mytest-Tqz13LJJ-py2.7/include/python2.7',
 'platlib': '/home/finswimmer/.cache/pypoetry/virtualenvs/mytest-Tqz13LJJ-py2.7/lib/python2.7/site-packages',
 'platstdlib': '/home/finswimmer/.cache/pypoetry/virtualenvs/mytest-Tqz13LJJ-py2.7/lib/python2.7',
 'purelib': '/home/finswimmer/.cache/pypoetry/virtualenvs/mytest-Tqz13LJJ-py2.7/lib/python2.7/site-packages',
 'scripts': '/home/finswimmer/.cache/pypoetry/virtualenvs/mytest-Tqz13LJJ-py2.7/bin',
 'stdlib': '/home/finswimmer/.cache/pypoetry/virtualenvs/mytest-Tqz13LJJ-py2.7/lib/python2.7'}
>>>

finswimmer on 29 Oct 2020

@finswimmer I have no idea where local is coming from, but like I said above, this is my setup.

I've installed the latest poetry in a python 3.7 virtualenv. I run all the poetry commands from this virtualenv. I can reproduce this issue on demand. I use the following to install the packages into a python 2.7 virtualenv:

poetry env remove 2.7

poetry env use 2.7

poetry install

The virtualenv that poetry runs in was created by version 16.7.10 on a Ubuntu 20 system.

rzuckerm on 30 Oct 2020

I have a self-contained docker environment that reproduces this issue in this GitHub repo. The docker environment is as close to our production environment as possible. It is a Ubuntu 18.04 environment with the latest poetry installed.

To reproduce this issue, Just clone the repo and run make issue. This does the following:

Build the Ubuntu 18.04 docker image.
Run this script in the docker container which does the following:
- Create the python 2.7 virtualenv using poetry env use 2.7.
- Run a python script in the python 2.7 virtualenv that shows the system configuration using poetry run python -c '...'. You'll notice that there are paths that have the local directory that I believe are the root cause of this issue.
- Install packages into the python 2.7 virtualenv using poetry install -vvv.

rzuckerm on 31 Oct 2020

Feels like it would be worth removing _poetry_ from the equation and figure out if it is not caused by _virtualenv_ itself (or something else, deeper than _poetry_ anyway).

_[I know I write this on all tickets these days :D]_

I did a quick test (really did not dig deep, so it might be a false lead), I ran python -m sysconfig in multiple environments created with _virtualenv_:

$ virtualenv --version
virtualenv 20.0.34 [...]
$ virtualenv -p 3.7 venv37
# [...]
$ venv37/bin/python -m sysconfig
Platform: "linux-x86_64"
Python version: "3.7"
Current installation scheme: "posix_prefix"

Paths: 
        data = "/tmp/tmp.c9PgQTb1Zm/venv37"
        include = "/usr/include/python3.7m"
        platinclude = "/usr/include/python3.7m"
        platlib = "/tmp/tmp.c9PgQTb1Zm/venv37/lib/python3.7/site-packages"
        platstdlib = "/tmp/tmp.c9PgQTb1Zm/venv37/lib/python3.7"
        purelib = "/tmp/tmp.c9PgQTb1Zm/venv37/lib/python3.7/site-packages"
        scripts = "/tmp/tmp.c9PgQTb1Zm/venv37/bin"
        stdlib = "/usr/lib/python3.7"
# [...]
$ virtualenv -p 3.8 venv38
# [...]
$ venv38/bin/python -m sysconfig
Platform: "linux-x86_64"
Python version: "3.8"
Current installation scheme: "posix_prefix"

Paths: 
        data = "/tmp/tmp.c9PgQTb1Zm/venv38"
        include = "/usr/include/python3.8"
        platinclude = "/usr/include/python3.8"
        platlib = "/tmp/tmp.c9PgQTb1Zm/venv38/lib/python3.8/site-packages"
        platstdlib = "/tmp/tmp.c9PgQTb1Zm/venv38/lib/python3.8"
        purelib = "/tmp/tmp.c9PgQTb1Zm/venv38/lib/python3.8/site-packages"
        scripts = "/tmp/tmp.c9PgQTb1Zm/venv38/bin"
        stdlib = "/usr/lib/python3.8"
# [...]
$ virtualenv -p 2.7 venv27
# [...]
$ venv27/bin/python -m sysconfig
Platform: "linux-x86_64"
Python version: "2.7"
Current installation scheme: "posix_local"

Paths: 
        data = "/tmp/tmp.c9PgQTb1Zm/venv27/local"
        include = "/tmp/tmp.c9PgQTb1Zm/venv27/local/include/python2.7"
        platinclude = "/tmp/tmp.c9PgQTb1Zm/venv27/local/include/python2.7"
        platlib = "/tmp/tmp.c9PgQTb1Zm/venv27/local/lib/python2.7/dist-packages"
        platstdlib = "/tmp/tmp.c9PgQTb1Zm/venv27/lib/python2.7"
        purelib = "/tmp/tmp.c9PgQTb1Zm/venv27/local/lib/python2.7/dist-packages"
        scripts = "/tmp/tmp.c9PgQTb1Zm/venv27/local/bin"
        stdlib = "/tmp/tmp.c9PgQTb1Zm/venv27/lib/python2.7"
# [...]

Notice how the _installation scheme_ is posix_local for _Python 2.7_.

sinoroc on 31 Oct 2020

Potentially related:

https://github.com/pypa/virtualenv/issues/814

Seems like it might occur only on Debian/Ubuntu. Could explain why @abn and @finswimmer could not recreate the issue at first (don't know what the python:2.7 base image uses).

sinoroc on 31 Oct 2020

@sinoroc That issue is about 5 years old, and it is marked as "won't fix". Since it's unlikely that virtualenv will be fixed, can't something be done in poetry to create the necessary directories if they don't already exist? I have already suggested two places to fix this.

rzuckerm on 1 Nov 2020

@rzuckerm

If the issue I linked is indeed the root cause, my hope is that it will help _poetry_'s maintainers to figure out what could be done to fix it. Or on the other hand you could add your voice to hopefully get a fix in _virtualenv_. Actually you should raise these concerns against Debian/Ubuntu, they notoriously mess up many things for Python packaging. But really I am not even sure that we identified the root cause, have we?

Now what I really would like to encourage you to do is:

1. Install a real Python distribution on Debian/Ubuntu, do not rely on the ones that are installable from their repositories (they are notoriously hard to work with in some areas). It might well fix the issue for your use cases. Give one of these a try:

_deadsnakes_
_pyenv_
install from scratch
switch to a different Linux distribution
etc.

2. If you want to work with Python 2.7, get your hands dirty. Since it has reached its end of life (no scoop here), I feel like it is getting harder and harder to justify asking open source maintainers to help fixing issues related to _Python 2.7_. And I believe people really should up their game in matters of fixing those issues themselves, instead of relying on open source maintainers. You are already in a good position, since you seem to have found places where to add fixes. You could already file a (draft) PR, so that you can use your own fork for your uses case, and so that this fix can potentially be included after review and so on.

_[I am not a maintainer of these 2 projects (_virtualenv_, _poetry_). I am speaking for myself. So as much as I try to be accurate, I might be wrong.]_

sinoroc on 1 Nov 2020

@sinoroc Thanks for your feedback. I'll have to discuss this with my colleagues and see how to proceed.

rzuckerm on 2 Nov 2020

👍1

We've decided not to try to fix this in poetry. Instead, we have modified our build system to create the necessary sysconfig directories before invoking poetry install. That seems to work.

rzuckerm on 3 Nov 2020

👍1

Since we have a workaround. I'm closing this issue.

rzuckerm on 8 Nov 2020

Was this page helpful?

0 / 5 - 0 ratings