Poetry: Expose Poetry's virtualenv

[ambv]

If the user doesn't have an active virtualenv, Poetry is already creating and using a default per-project virtualenv behind the scenes. There is no easy way to get to it after it's created.

If you'd be willing to accept a PR to this effect, I'd be happy to contribute the following:

• add a poetry shell command equivalent to pipenv shell
• make poetry add/remove/install output the path to the virtualenv that it's using if there is no active virtualenv

I know that your current available alternative is poetry run. My problem with it is that it gets verbose fast. poetry run python -m X instead of just X in an activated virtualenv.

[moigagoo]

It sounds reasonable, but I have concerns about Windows compatibility. Pipenv seems to discriminate Windows users and AFAIK pipenv shell doesn't work with pwsh or cmd. If you're going to implement poetry shell, please don't forget about Windows.

[ambv]

I don't use Windows so I'll leave that part as an exercise to the reader.

[moigagoo]

I'll be happy to contribute then :-)

[sdispater]

The output of the virtualenv path is already implemented but only when you use the add/remove/install commands in verbose or higher mode (-v option).

Regarding the poetry shell, if someone to tackle this, I'd gladly review a PR :-)

[cauebs]

I'd like to give this a go, but I'll need some help (probably my first code contribution to any project). I believe you can't inject a script into the running shell from python, so we'll need to spawn a new shell. How should I approach this? subprocess.run or something, right? I'm just very afraid that I don't know much about how terminals and shells work and may miss some edge cases. Anyone willing to work with/mentor me?

[ethanhs]

FWIW, I'd be happy to test/fix/implement things on Windows. I really like poetry and am keen for this functionality.

[jgirardet]

@cauebs look at run.py in commands to starts.

[cauebs]

Thanks for the tip, @jgirardet. I gave it a go, it seems to work and I opened #201.

[jaroel]

Here's a workaround to get it:
$ poetry run which python
/Users/roel/Library/Caches/pypoetry/virtualenvs/koei-py3.6/bin/python

[sobolevn]

I will also gently remind about a possibility to have a local .venv folder for dependencies.
pipenv has PIPENV_VENV_IN_PROJECT=true option which allows that.

[sdispater]

@sobolevn This is already supported, see #213

[cauebs]

@moigagoo @ethanhs I think #201 is close to being merged, but it only features provisional support for Windows (which hasn't even been tested yet). Would you take a look and let me know how I can improve it? It would probably be important to know how to properly which program to run, if not cmd, and which flags are necessary.

[moigagoo]

@cauebs Thanks! cmd is definitely not the best choice. I can hardly imagine anyone using it as a regular shell. Powershell is currently the de-facto proper Windows shell.

I think poetry shell should spawn the same shell it was called in. Calling cmd makes it pretty useless on Windows.

[ethanhs]

@moigagoo I use cmd as my main shell, so I would strongly disagree.

[moigagoo]

@ethanhs Sorry, I didn't mean to offend. Anyway, spawning the shell the command was called in seems like the right choice.

[cauebs]

I could try implementing this, but I really don't have a way to test it.
Pipenv (or Pew) implements this manually, without adding extra dependencies, but it's quite ugly...
https://github.com/pypa/pipenv/blob/master/pipenv/patched/pew/_win_utils.py

[cyniphile]

Having some problems getting this to work. Running poetry shell does spawn a new shell, but not with the proper venv activated (which python3 is still system python). Debugging shell.py, I looked at what self.venv.execute(shell.path) was doing, and shell.path is simply '/bin/zsh'...so it's just starting a new zsh shell inside the current one. a) I don't really want a nested zsh shell, and b) shouldn't there be some kind of venv/bin/activate thing going on instead?

I tried subbing in self.venv.execute(self.venv.venv.as_posix() + '/bin/activate') but got a permission denied error.

This is on mac with zsh. Just installed poetry today.

[Mokubyow]

I'm seeing the same issue as @cyniphile. I'm also using zsh.

[seansfkelley]

@cyniphile @Mokubyow I was also seeing your issue with poetry shell continuing to use system Python.

In my case this issue was caused by configuration differences in ~/.profile/~/.zprofile versus ~/.zshrc and $PATH modifications. More details on the differences at this SO post.

Relevant bits: rcs are run for every shell instantiation, but profiles are only run by the login shell. If you move your unconditional path modifications (like adding /usr/local/bin for Homebrew) to a profile instead of an rc, they will run once but will permit commands like poetry shell to override them at a later time. If you put it in an rc, poetry shell will override it and then you will re-override it unconditionally immediately after. In my case, Python was installed by Homebrew, so that re-overriding meant that Homebrew Python was always selected first even when the virtualenv path was present.

I don't know how much of this issue can be resolved on Poetry's part, but I resolved it to my satisfaction by moving several path edits into ~/.profile. Alternately, you can do something conditional like at https://github.com/sdispater/poetry/issues/172#issuecomment-401554154.

[cyniphile]

thanks, @seansfkelley that was my issue. I was actually loading .profile in .zshrc (due to my own misunderstanding of how the different shell config files work).

In other news, it would be nice for poetry shell to activate the virtual environment so I can get the nice parentheses showing the env in my command line. I only know I'm in a virtualenv shell from running which python

[cauebs]

@cyniphile We don't change the prompt at the moment because it would require specific implementations for different shells. If someone opens a PR with an initial implementation, I'm sure it would be very welcome :)

[cyniphile]

@cauebs with the caveat that I'm not super knowledgable in this domain, I tend to disagree with the way poetry shell is implemented. Creating a new shell inside the current one seems unnecessary and causes the problems I and others ran into with zsh (even though they were partially our own fault in a way as well).

Is it not viable to simply activate the virtualenv poetry creates? I don't see what is shell specific about simply calling source path/to/pyproject/python/activate.

Thanks for your contribution here, not trying to be one of those ungrateful open source users. I would be happy to try to figure out and activate based solution, but I want to know why you didn't go down that route?

[cauebs]

@cyniphile

To be quite honest, I understand that it may seem simple, but as far as I know you can't alter the environment for a parent process.

We could do what Pipenv does, which involves running the shell in a subprocess just as well, but then making it run the source command automatically (and that requires some very complex magic implemented by pexpect's ptyprocess). Now, I don't currently have the time to delve into this (https://github.com/pexpect/ptyprocess/blob/master/ptyprocess/ptyprocess.py#L178-L339), especially taking into account that all we would get is the prompt.

So, in a way, you're perfectly right: we could do this by "simply" sourcing the activation scripts. However, it's not an easy task. Rest assured that if we eventually run into any problems with the current approach, I'll be the first one to give this alternative a go.

And you didn't sound ungrateful at all. Open source relies on criticism and feedback just as much as it does on code contributions. It's always good to reevaluate our past decisions, so thanks for that :)

[sobolevn]

I have already suggested a simple solution in some other issue, but I will repeat it.
We can go with source "$(poetry venv)".

Why do I like it?

1. It straight simple, there's nothing more to remove
2. It is platform and shell independent: some shells use source, while raw sh uses .. But we do not need to care
3. It is similar to the existing solution, that most of users are used to
4. It exposes venv path, so you can use it in any way you want

Are there any drawback in this solution that I do not see?

[sersorrel]

(what does the poetry venv command do? I can't find documentation for it anywhere.)

It is platform and shell independent

Not true; there are different activate scripts per shell (activate for bash, activate.csh for csh, activate.fish for fish).

some shells use source, while raw sh uses .. But we do not need to care

Also not true; POSIX only specifies ., source is not required and is not supported in some shells.

[sobolevn]

Sorry, for not describing it well enough. poetry venv does not currently exist.
This command just outputs the path of your virtualenv script location. That's the only API that is going to be provided by poetry.

So, you can use this path in any way you want on your platform. You can use source, ., etc to activate this script. Just like it currently works for the venv module.

Examples:

$ poetry venv
/User/path/.venvs/my_project/bin/activate

$ source "$(poetry venv)"
Activating "my_project" venv

$ (my_project)

That was my point. I hope that it makes sense.

---

Or it can output just the base path for the venv base directory. So you will have to write source "$(poetry venv)/bin/activate" in order to actually activate it.

[cauebs]

You are both missing my point. How do you suggest we run source for the shell?

I'll reiterate:

• >you can't alter the environment for a parent process
• >We could do what Pipenv does, which involves running the shell in a subprocess just as well, but then making it run the source command automatically (and that requires some very complex magic implemented by pexpect's ptyprocess).

I don't see how this would improve poetry shell in any way other than delegating the change in the prompt to the venv scripts. For that, we could just as well translate the scripts' actions to Python (which environment variable to change, etc).

About the venv path, it's already available in the VIRTUAL_ENV variable. Even if we want to expose it in some other way (which I did in #278, as a proof of concept), it's an information Poetry already has, so your suggestions are far beyond the point.

[cauebs]

By the way, we are drifting off the topic proposed by this issue. I think it has already been resolved and could be closed. If you have any other specific suggestion or complaint, feel free to look the issues pertinent to it or open new ones ;)