Poetry: Option to create virtual environments in the project root (.venv)

[jacebrowning]

I'm considering migrating some of my projects from pipenv to poetry. I would like to be able to tell poetry to create virtual environments at ./.venv/ to support Makefile target dependencies, CI build caching, and automatic IDE support.

I added a feature to pipenv to detect and use virtual environments located at ./.venv/ based on the mere presence of a directory of that name. It looks like poetry nearly has the same behavior:

https://github.com/sdispater/poetry/blob/87c4aaf9cadbbb6c5657101d715b5a4e1a8d9a05/poetry/utils/venv.py#L55-L59

but I noticed that if .venv is an empty directory (in order to let the tool populate it) poetry seems to install dependencies into my global Python instead.

Would you consider a PR to add another case to this logic to create virtual environments in the root of a project? This should allow poetry to handle all of the following scenarios:

• Use the currently activated virtual environment ($VIRTUAL_ENV set)
• Use the exiting local virtual environment (./.venv/<bin>/python exists)
• [NEW] Create a new local virtual environment (./.venv/ is empty)
• Use the previously cached virtual environment
• Create a new cached virtual environment

[jacebrowning]

Alternatively, an option in pyproject.toml to specify the virtual environment location would also be great!

[sdispater]

I might add a setting that can be set via the config command to tell poetry to create the virtualenv inside the project directory.

The only downside to that is that you can only have one virtualenv by project.

[buriy]

@jacebrowning isn't there a config option already? https://github.com/sdispater/poetry/blob/87c4aaf9cadbbb6c5657101d715b5a4e1a8d9a05/poetry/utils/venv.py#L64
Not a command-line one, though, yet.

[jacebrowning]

@buriy How might I use that? I'm having trouble understanding the documentation here:

$ poetry config -h
Usage:
  config [options] [--] <key> [<value>]...

...

 To add a repository:

     poetry repositories.foo https://bar.com/simple/

 To remove a repository (repo is a short alias for repositories):

     poetry --unset repo.foo

$  poetry virtualenvs.path .venv

[CommandNotFound]
Command "virtualenvs.path" is not defined.



md5-160b1f677e4954f5a5e8dfa146340919



$ poetry config virtualenvs.path .venv

[ValueError]
Setting virtualenvs.path does not exist

config [--list] [--unset] [--] <key> [<value>]...



md5-160b1f677e4954f5a5e8dfa146340919



$ poetry config settings.virtualenvs.path .venv

[ValueError]
Setting settings.virtualenvs.path does not exist

config [--list] [--unset] [--] <key> [<value>]...

[buriy]

@jacebrowning I guess if you manually create the config.toml file, it would work, but there's no CLI support for that right now. Please, could you try if that works?

[sdispater]

@buriy @jacebrowning This setting is here to tell poetry where to create every virtualenv so it wouldn't work for this use case. That's why I am thinking of adding a new setting to tell poetry to create the virtualenv for the current project inside the .venv directory.

[bertjwregeer]

Having a .venv directory would be great since many tools look there for formatting tools and the like, specifically I am thinking of ALE for Vim.

[Victor-Savu]

Just wanted to point out the similarity to the approach Rust's package manager cargo has. cargo creates a directory target in the project root where it places all the dependencies and compilation artifacts. The proposed .venv directory seems to go in the same direction.

[kelseasy]

Python packages can/have to support both python2 and python3, whereas you rarely need multiple rust versions to compile with.

Something along these lines in the pyproject.toml being used to create venvs could go a long way:

[tool.poetry.virtualenvs]
venv = { python = "~2.7" }
venv3 = { python = "^3.5" }

[Victor-Savu]

@kelseasy I just thought it would be fun to share this: https://pythonclock.org/

[buriy]

Usually tox is used if you need to test several versions, and I haven't
seen anybody who needs to have several venvs for a reason other than that.
Only for upgrade maybe but then you can set up 2 project folders, each with
its venv.
Support for several venvs is ok, but if having only one venv simplifies
things greatly, and doesn't hurt anyone, why not introducing this limit?

ср, 23 мая 2018, 5:07 Kelsey Z [e-mail]:

python packages can/have to support both python2 and python3, whereas you
rarely need multiple rust versions to compile with.

Something along these lines in the pyproject.toml being used to create
venvs could go a long way:

[tool.poetry.virtualenvs].venv = { python = "~2.7" }.venv3 = { python = "^3.5" }

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sdispater/poetry/issues/108#issuecomment-391156323,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABShBuar33_GQtedJC73OtlNG3yLAyfks5t1IwPgaJpZM4UAPUe
.

[sdispater]

The new settings.virtualenvs.in-project setting is now available. You can activate it with:

poetry config settings.virtualenvs.in-project true

[Imaclean74]

This is great - glad to be able to set this. Question though - does it make more sense for local .venv to be the default ? This is the first thing I changed when I started using pipenv - since it also wants to create the venv under user settings. More than likely - I'm missing some fundamental reason that using user settings is a better option - but arguments for defaulting to local .venv might be :

• Obvious where those deps went - you have new directory right there
• This is how npm works - so familiarity for devs with any java script experience. Devils advocate - gradle uses a shared cache - and I don't know what cargo ( one inspiration for poetry ? ) uses.
• familiar for python devs used to the standard virtualenv workflow
• the tooling issue raised by a few others. VSCode for example will automatically find a local venv - but has no way of knowing which venv under /Users//Library/Caches/pypoetry/virtualenvs to look for

[Atrox]

Sorry for commenting on an old issue, but is there a way to set this option for a project (in pyproject.toml) and not per machine with poetry config?

[b-long]

@Atrox This looks like an open issue: https://github.com/sdispater/poetry/issues/618

[Cito]

I noticed in v1 beta it only works without the settings.-prefix: poetry config virtualenvs.in-project true

[bangseongbeom]

@Cito, you mean poetry config --local virtualenvs.in-project true?

[Cito]

@bangseongbeom Yes, sorry, corrected that already.

[jwiede]

If you had a project created using virtualenvs.in-project=false, and then subsequently set the config setting to true, is there any way to get poetry to copy the venv contents from its prior venv location to the "new" location?

[heynemann]

@jwiede yes you can. Just do the following:

$ poetry env list

Find the name of the virtual environment you currently have.

$ poetry env remove

This will delete the Venv. Now just do poetry install and it will work out.

[cgarciae]

It would be nice if this was the default so you don't end up with zombie environments when you delete a project's folder.

[joshfriend]

I've read numerous issues for both poetry and pipenv about placing the venv in the project directory by default and cannot recall a single reason why that is a bad idea.

[Cito]

@joshfriend One disadvantage is that you can easily create monster size backups with the ".venv" und "node_modules" sub-directories of all your various projects, which is completely unnecessary because you can easily recreate them. It took me quite a while to find a backup program that allows to generally ignore sub-directories with certain names, and not just a list of fixed paths or files with certain extensions. If the ".venv"s are all in a central place, you can more easily exclude them from backups.

[cgarciae]

@joshfriend This is a good use case for global directory feature but not a reason why it should be the default. I think 90% or more of the users would benefit with the more transparent and maintainable local strategy.

The local .venv folder is even recognized by vscode automatically while the with the global strategy its a bit tricky to setup at first. In other words, using the local .venv folder improves user experience a lot for those starting to get familiar with poetry.

[cgarciae]

@sdispater local (in-project) by default would be an amazing feature for 1.0.0!

[joshfriend]

1.0 has already been released 😄

I'm glad @Cito mentioned the backups reason, personally I don't mind having 100 venvs in my backups, but thats not true for everyone ;)

I still don't think thats a good enough reason to keep in-project .venv location off by default.

[Cito]

@joshfriend to be clear about that - I also like to have my .venvs local and think that should be the default. The backups only became really problematic for me once I started to have also node_modules directories in my projects, which serve a similar purpose in JavaScript, are always local and are notoriously known as heaviest objects in the universe.

[Cito]

Local .venvs are also automatically recognized by PyCharm.

[gsemet]

With poetry > 1.0.0, the command to set in-project venv is:

poetry config --local virtualenvs.in-project true

[mikelnrd]

And VSCode

Local .venvs are also automatically recognized by PyCharm.

[ChristianSauer]

There is also the problem that shared configuration in VSCode does not really work without local envs.
The shared configuration takes a path - and this path is different for every user if the .venv is not in the project directory.
Honestly, bkacups are not even on my radar, I use git and a matching .gitignore.

[gsemet]

Why not using a $ variable ?

[b-long]

I'm surprised that the poetry.toml file hasn't been mentioned on this thread. I place a file called poetry.toml into all of my Python projects:

[virtualenvs]
create = true
in-project = true

This produces a local .venv/ folder, and you can configure your editor to point to it using a relative path. For example, with VSCode, you could place this in settings.json:

"python.pythonPath": "${workspaceFolder}/.venv/bin/python"

[joshfriend]

I've been reading and commenting on threads like this both in Poetry and Pipenv for well over 2 years now and it seems pretty clear to me that the most beneficial default is having in-project be true. I think a LOT of people don't care either way, probably because they don't understand the benefits of in-project, whose docs mention zero reasons for or against using it. Then there are the users like myself who actually do care, because we want our development tools to integrate easily with each other.

The remaining users have a few (valid) edge cases that they care about which I'm fairly certain make up the minority:

1. Reducing backup size and clutter
2. Another case I saw on a different thread was about using poetry/pipenv locally and with a docker container simultaneously. in-project would cause the venv to be shared with two incompatible platforms, whereas the hidden venv option takes care of that

I don't think either is difficult to work around, and I think the benefits of compatibility with PyCharm/VSCode/etc should carry more weight in the decision of what should be the default.

Further, to what @b-long is saying above me, I _knew exactly what I wanted poetry to do for me_ and I _still_ found it difficult to find the CLI command to set that flag or write it manually in pyproject.toml.

in-project=true as the default would be amazing for the development experience. I'm 🙏 _begging_ y'all to change it 😆

[mfoglio]

I'm surprised that the poetry.toml file hasn't been mentioned on this thread. I place a file called poetry.toml into all of my Python projects:

[virtualenvs]
create = true
in-project = true

This produces a local .venv/ folder, and you can configure your editor to point to it using a relative path. For example, with VSCode, you could place this in settings.json:

"python.pythonPath": "${workspaceFolder}/.venv/bin/python"

But does that the path configuration works too? Because it seems to be ignored.
Source: https://python-poetry.org/docs/configuration/#virtualenvspath-string