Poetry: Poetry removes some of its own dependencies when running `poetry install --no-dev`

[Celeborn2BeAlive]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10
• Poetry version: 1.1.3
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/Celeborn2BeAlive/a94feac94d03a4057a1a11095de7bc91

Issue

When Poetry 1.1.3 is installed in the virtual environment of the project (.venv folder), and configured to use virtual env in project (in-project = true), then when I run poetry install --no-dev it removes some of its own dependencies, which results in a broken poetry.

Here are the steps I follow in a folder containing the pyproject.toml linked, that only have dev dependencies:

python -m venv .venv
source .venv/Scripts/activate
pip install poetry
poetry install --no-dev # The first one does nothing to the venv and just write the lock file, which seems correct
poetry install --no-dev # The second one uninstall some poetry dependencies

The list of uninstalled dependencies is:

poetry install --no-dev
Installing dependencies from lock file

Package operations: 0 installs, 0 updates, 12 removals

  • Removing appdirs (1.4.4)
  • Removing certifi (2020.6.20)
  • Removing chardet (3.0.4)
  • Removing distlib (0.3.1)
  • Removing filelock (3.0.12)
  • Removing idna (2.10)
  • Removing packaging (20.4)
  • Removing pyparsing (2.4.7)
  • Removing requests (2.24.0)
  • Removing six (1.15.0)
  • Removing urllib3 (1.25.11)
  • Removing virtualenv (20.0.35)

After that, poetry does not work anymore, here is a traceback:

poetry --version

Traceback (most recent call last):
  File "C:\Users\laure\.pyenv\pyenv-win\versions\3.7.9\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\laure\.pyenv\pyenv-win\versions\3.7.9\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\projects\c2ba_cookiecutters\python_cookiecutter\_test\.venv\Scripts\poetry.exe\__main__.py", line 4, in <module>
  File "d:\projects\c2ba_cookiecutters\python_cookiecutter\_test\.venv\lib\site-packages\poetry\console\__init__.py", line 1, in <module>
    from .application import Application
  File "d:\projects\c2ba_cookiecutters\python_cookiecutter\_test\.venv\lib\site-packages\poetry\console\application.py", line 7, in <module>
    from .commands.about import AboutCommand
  File "d:\projects\c2ba_cookiecutters\python_cookiecutter\_test\.venv\lib\site-packages\poetry\console\commands\__init__.py", line 4, in <module>
    from .check import CheckCommand
  File "d:\projects\c2ba_cookiecutters\python_cookiecutter\_test\.venv\lib\site-packages\poetry\console\commands\check.py", line 2, in <module>
    from poetry.factory import Factory
  File "d:\projects\c2ba_cookiecutters\python_cookiecutter\_test\.venv\lib\site-packages\poetry\factory.py", line 16, in <module>
    from .packages.locker import Locker
  File "d:\projects\c2ba_cookiecutters\python_cookiecutter\_test\.venv\lib\site-packages\poetry\packages\__init__.py", line 2, in <module>
    from .locker import Locker
  File "d:\projects\c2ba_cookiecutters\python_cookiecutter\_test\.venv\lib\site-packages\poetry\packages\locker.py", line 36, in <module>
    from poetry.utils.extras import get_extra_package_names
  File "d:\projects\c2ba_cookiecutters\python_cookiecutter\_test\.venv\lib\site-packages\poetry\utils\extras.py", line 7, in <module>
    from poetry.utils.helpers import canonicalize_name
  File "d:\projects\c2ba_cookiecutters\python_cookiecutter\_test\.venv\lib\site-packages\poetry\utils\helpers.py", line 10, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

To make it work again I have to reinstall it with pip.

[Celeborn2BeAlive]

It also happens on Linux, in the python:3 docker image, here is a log on Gitlab CI: https://gitlab.com/c2ba_cookiecutters/python_cookiecutter/-/jobs/805609953

[finswimmer]

Hello @Celeborn2BeAlive,

this is an expected behaviour and the reason why poetry should not be installed in the same environment as the project. poetry doesn't know about about packages, it hasn't installed by itself. When running poetry install --no-dev poetry will remove dependencies that are needed by a dev dependency. And this can be dependency that poetry needs itself.

Solutions: Use the recommended installer for poetry or install poetry via pipx.

fin swimmer

[Celeborn2BeAlive]

@finswimmer Alright thanks for this information.

One issue I had with poetry installed with pipx is that it was "bound" to the python of its dedicated virtual env. In my case it was 3.7.x, so I was unable to handle projects with other versions of python :/

I guess I'll have the same behavior using the recommended installer but I can try, I'll take a look at the script before.

But anyway another option is to have one virtual env just for poetry, and one for my project.

[finswimmer]

One issue I had with poetry installed with pipx is that it was "bound" to the python of its dedicated virtual env. In my case it was 3.7.x, so I was unable to handle projects with other versions of python :/

For that use case poetry env use exists. Before running poetry install type poetry env use /path/to/python/of/choice.