Poetry: How can I sync the packages on pyproject.toml(or poetry.lock) and on virtualenv?

[moonkwoo]

• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have searched the documentation and believe that my question is not covered.

• Poetry version: 0.12.10
• OS version and name: macOS 10.14.2 Mojave

Question

For example, I remove a package(e.g Arrow) from pyproject.toml by poetry remove arrow and I see the package is removed from pyproject.toml, poetry.lock and my virtualenv. And then I deploy pyproject.toml and poetry.lock.
My teammate download these files and try to install by poetry install. However arrow package is NOT removed from teammate's virtualenv. I want to sync them (pyproject.toml and virtualenv) like pipenv sync. How can I solve this problem?

[jgirardet]

your teammates have to do a poetry update

[moonkwoo]

@jgirardet I try to poetry update, but the arrow package is not removed from teammates's virtualenv :(
When he do poetry update, the message was Nothing to install or update.

[garyo]

I'm not sure this would be a good idea -- at least not by default. Perhaps a poetry sync command that would remove anything in the virtualenv which isn't in poetry.lock but for many people that's going to be seen as somewhat dangerous. I think your best option for now is to recreate the virtualenv.

[moonkwoo]

@garyo Thank you so much :) I will try to ecreate the virtualenv.

[rpdelaney]

but for many people that's going to be seen as somewhat dangerous.

@garyo Could you expand on what danger would attach to this?

I think your best option for now is to recreate the virtualenv.

Is there a way to do that with poetry? For instance, in pipenv there is pipenv --rm. With my growing but still inadequate knowledge of how best to manage a venv in python, the best I have in poetry is basically type -a flake8 or such to find the venv path and then rm it. It has a smelly feeling, that this can't be the best way.

[garyo]

@rpdelaney sure -- anything you install manually in the virtualenv (e.g. with plain pip install) will get deleted. Or if you've symlinked some things around for dev/test purposes, those symlinks will get deleted. Of course if you only do things "properly" it'll be fine, but any time a command silently wipes a subdir you know someone will lose something, especially if the command name is something as generic as "sync". Maybe like git clean (which is similarly "dangerous") it should require a --force option to ensure you know what you're doing. Or maybe call it poetry remove-untracked or something.

[rpdelaney]

Thank you for that, it helps.

I typically have this problem when switching between git branches where one branch has a dependency that another does not. Checking out another branch does not "check out" another venv, which can lead to my venv having broken dependencies vis the code that's on disk. In pipenv an extra sync --dev is a relatively easy solution.

Sometimes I install things in the venv that aren't an explicit project dependency. I don't like installing things with pip outside of a venv so this might include extra linters, another REPL I like, things like that. So I get that concern also. Using symlinks to modules is something I hadn't heard of but it's interesting.

I think the point I am still resting on is that "go find the venv and rm -r it yourself" doesn't seem any less smelly or dangerous as a solution. I'm probably going to wind up storing my venvs in the project directory when using poetry, which imho isn't ideal either. I'm not sure what the ideal solution would be, but even a poetry get-venv-path command could enable an alias like rm -r "$(poetry get-venv-path)" or such without feeling like we have to give poetry the responsibility of ensuring it is deleting the right files.

[finswimmer]

but even a poetry get-venv-path command [...]

In the latest preview there is already such a command: poetry env info -p

[mjlbach]

I've run into situations in which this would be useful, such as switching between the tensorflow and tensorflow nightly builds which share the same namespace. pip-tools has pip-sync which implements this functionality.

[martinka]

I think this would be hugely useful. And actually seems like what you would want if you want reproducible builds. I have the situation where a package has been flagged for security issues, was removed, but is still hanging around on build servers. poetry update is not an option as that modifies the lock file ( i.e. no longer reproducible). Currently I am force to remove the virtual env, which results in very long build times.

[mjlbach]

@martinka Does this PR not address your concerns? https://github.com/python-poetry/poetry/pull/2172

[finswimmer]

Yep, this is addressed by #2172