Poetry: Add awareness of pip installed packages

[ketozhang]

Poetry is not aware of accidental/purposeful use of pip to install packages

$ poetry init
$ poetry add tqdm
$ poetry shell
$ pip install numpy # e.g., user did this accidentally 

Actual

$ poetry lock && poetry install
$ poetry show
tqdm 4.50.0 Fast, Extensible Progress Meter

Expected

$ poetry lock && poetry install
$ poetry show
tqdm 4.50.0 Fast, Extensible Progress Meter
numpy 1.19.2 NumPy is the fundamental package for array computing with Python.

I understand this is very difficult to support currently and the current design recommends user to avoid pip in favor of installing via poetry. However, in peculiar cases (e.g., syntax conventions), sometimes install via poetry is not possible/easy and pip is preferred to be used.

[malcolmgreaves]

~This is actually expected behavior: nothing is wrong here.~

~Your project will be installed in an isolated virutalenv. When you do pip install numpy, that installs it to the system-wide environment -- not the project environment.~

~You don't want to have your project dependencies intermingled with your system, or other project, dependencies. This is a core design feature of poetry: it is a good thing.~

EDIT -- Nope, I'm totally wrong. I didn't see the poetry shell 🤦‍♂️

[ketozhang]

My apologies, I forgot to mention the pip install is ran inside the virtual env hence this behavior exist inside the project environment. Will edit the example.

[n8sty]

I think the suggested behavior would actually add a fair bit of difficulty to working on a poetry managed project. Sometimes I'll pip install ipython into a poetry managed virtual environment because I want to use a fancier REPL or pip install a dependency to play with some additional functionality that I may or may not want added to as a project dependency. If each of these actions automagically added a dependency, I'd have to remember to revert pyproject.toml and poetry.lock. I like the current behavior because it's explicit. When I add a new dependency I poetry add, and that's it. I think this proposal might lead to a fair bit of commit reverts because dependencies were erroneously added.

[ketozhang]

It looks like what I'm really looking for is to be able to undo pip install operation(s) without needing to know its name similar to pipenv's clean command. Unfortunately poetry install is not able to restore the environment.

@n8sty, in your case your python environment will not be representative by your lockfile or pyproject.toml which I believe poetry discourages.

[finswimmer]

It looks like what I'm really looking for is to be able to undo pip install operation(s) without needing to know its name similar to pipenv's clean command.

$ poetry install --remove-untracked ?

[malcolmgreaves]

@ketozhang My apologies -- I totally did not see the poetry shell and completely misunderstood!

[ketozhang]

No worries, as I've edited that in after your comment so your confusion was my fault

[ketozhang]

$ poetry install --remove-untracked ?

@finswimmer
Thanks that should satisfy my use case! Unfortunately, you must run it after the environment has been activated. It has weird behavior. Needing to activate the environment is not expected from poetry commands.

I've missed this having outdated poetry. Is it possible to make this behavior the default when running bare poetry install (maybe alias it to poetry [clean|sync]? My expectation is always "Python environment should always match the environment specification file". I wonder if this is a golden rule for poetry?

[finswimmer]

Is it possible to make this behavior the default when running bare poetry install (maybe alias it to poetry [clean|sync]?

There is already a feature request for this: https://github.com/python-poetry/poetry/issues/2370 We haven't finally discussed it until now.