Pipenv: Use venv instead of virtualenv if available

[jpadilla]

I'm wondering if it's worth using venv if available instead of virtualenv. That would also skip having to install virtualenv in the first place.

[kennethreitz42]

i don't think so, i prefer using the virtualenv command personally

[uranusjr]

Maybe it’s possible to make both options co-exist though? Casually browsing through the source I don’t see anything specifically coupled with Virtualenv except the command used to create the env. Virtualenv can always be the default, but the user can (for example) pass --venv to creates .venv with python3 -m venv instead of virtualenv.

[jpadilla]

I'd honestly be relying solely on pipenv and wouldn't care if virtualenv or venv was been used, but figured we could leverage builtins when available.

[bulletmark]

I urge @kennethreitz to reconsider his comment here. Using python3 -m venv instead of virtualenv is recommended by the standard documentation. I am personally put off using pipenv because it uses virtualenv. I only use Python 3 and only use venv nowadays.

[uranusjr]

@bulletmark Now that Pipenv delegates almost all Virtualenv management to Pew it is even more difficult to switch, but there is plan for Pew to support venv however (berdario/pew#67) so there is still chance. If Pew can get it working it would be much easier (and make more sense to me at least) to implement venv support in Pipenv.

[kennethreitz42]

@uranusjr sounds great!

[kennethreitz42]

I've personally never used venv, hence my apprehension towards it. I'm sure it's good, though.

Things are working well, right now. No need to change things that aren't broken.

[dhouck]

I eventually found this thread because my system is broken.

If you create an environment with virtualenv in Python 3.X, and then upgrade to Python 3.Y, the virtualenv is broken because you know longer have the shared libraries Python needs. Using venv you can just run python3 -m venv --upgrade.

[kennethreitz42]

This would be a nice feature.

[darrenhaken]

This would definitely be nice and feels inline with the direction of Python

[divad]

Not using venv (and instead using virtualenv) is a total showstopper for me - I cannot use Pipenv. I cannot use virtualenv because of its frustrating behaviour of copying the python binary and yet depending on shared libs - as dhouck found above to his displeasure above. This breaks virtualenvs all the time (whenever the system python is updated, leaving the binary dangling), and needlessly copies the python binary with absolutely no benefit.

If pipenv is the future of Python, it should default to using venv on Python 3 (and warn when not using it). Its built into Python 3, and its the recommended tool. Its very strange that Pipenv is also now a recommended tool but it doesn't use venv.

venv by nature of being part of Python itself has access to the internals of Python which means it can do things the right way with far fewer hacks. For example, virtualenv has to copy the Python interpreter binary into the virtual environment to trick it into thinking it's isolated, whereas venv can just use a configuration file that is read by the Python binary in its normal location for it to know it's supposed to act like it's in a virtual environment. So venv can be thought of virtualenv done right, with the blessing and support of the Python developers.

Source: https://www.reddit.com/r/learnpython/comments/4hsudz/pyvenv_vs_virtualenv/

[uranusjr]

I’ve started berdario/pew#173, but haven’t had much time to make progress. As mentioned above, Pipenv will automatically support this once Pew does. I’d like some help if anyone interested in bringing venv to Pipenv. The main issue right now would be to make the tests pass.

[uranusjr]

Quick update on venv support: It won’t happen, at least in near future. venv is simply not good enough to make this happen. If you really care about this, head over to the Python bug tracker and lend a hand. Otherwise, stick to virtualenv.

[divad]

venv is the recommended tool for Python, and it isn't broken, like virtualenv is (re: copying the python binary then breaks when python libs update). The bug you linked to is about combining venv and virtualenv, which isn't what we're asking for?

[uranusjr]

Unfortunately, it is extremely common for tools to build on virtualenv. Travis, for example, uses virtualenv to create a build environment for you, as mentioned by the original reporter. It is also the foundation of Pipsi, which if one of the recommended way to install Pipenv. If venv can’t work well inside it. It is in a lot of places without you knowing. Venv support would be half broken and bite you in unexpected ways if it can’t work well with virtualenv.

[divad]

I understand that, but at this point Pipenv is recommended, and venv is recommended, and they're sort of in conflict. Its a very weird state.

[uranusjr]

I totally agree it is in a weird state, that’s why I’m trying to fix it :) I’m not saying it won’t be fixed, but it needs upstream changes to happen first, and there’s nothing I can do until they happen. I post the link not as an excuse to stop working on it, but to let people know where to put the efforts first to help improve this situation.

[OmeGak]

Maybe it wouldn't be too crazy to keep this issue open then?

[kennethreitz42]

This is a pew problem, not a pipenv problem.

[cosven]

venv is simply not good enough to make this happen. If you really care about this, head over to the Python bug tracker and lend a hand.

It is not a problem of venv, but virtualenv. I have post some explanation under the issue tracker.
cc @uranusjr

[cosven]

There is a wordaround ->
Use venv to create a virtualenv first, and then run pipenv install . For example:

mkdir -p /tmp/try
cd /tmp/try
python3 -m venv .venv
pipenv --venv  # /tmp/try/.venv
pipenv install xxx

🙈

[dalai4git]

Just a note:

The workaround above mostly works, but pipenv run ... fails for me because there is no activate_this.py under .venv/bin. If issue 21496 is to be believed, there is no plan on adding this file to venv. So even if pew fixes this, pipenv would likely still have a problem.

[odiak]

Using virtualenv, using system tk module in macOS is impossible. But using venv, it is possible.
I strongly want to use venv with pipenv (may be optionally).

See also: #1416

[uranusjr]

Not to sound cynical, but I strongly want someone to actually write the code to support this if they really care so strongly. I’ve met numerous people saying they feel “strongly” about this, but as far as I know I am the only one to actually do anything. And I failed. This is not as simple as you think. If you think you can do better, do something. If you don’t want to, please take what you can get.

[techalchemy]

(kenneth and myself and probably erin and nate also tried to do this with venv much earlier in the project and also failed due to compatibility reasons)

This would be nice, everyone agrees. I'd love to see it working across platforms and python versions.

[tyrion]

I would like to point out an other reason why venv would probably be a better choice than virtualenv:

one of the uglier aspects of virtualenv's implementation is that it has to have its own copy of the site module, which is used for all virtualenvs regardless of which version of Python they are created with.
-- from https://github.com/pypa/virtualenv/issues/228#issuecomment-4165148

This means, for example, that rlcompleter is disabled by default and that is a huge usability loss in my opinion.

Moreover since this is an issue that impacts multiple of your users, I would suggest keeping it open until the problem is fixed, even if you are waiting for some changes upstream to happen.

[techalchemy]

Again. We know it would be better. It’s not actually possible. What do you suppose has a more negative impact, completely breaking functionality for a large segment of users, or having some users have to turn on readline features?

I’m sure this is inconvenient, it’s also inconvenient that there only seem to be people demanding this and yet nobody who will offer an implementation that works. We have tried. Until something changes drastically we have no plans to support venv and therefore no plans to open this issue.

[FranklinYu]

@techalchemy If I understand correctly, pypa/virtualenv#1095 is the correct place to discuss this?

[techalchemy]

Seems right, I saw tox has a workaround, so maybe that would work if someone ported it over

[doganmeh]

pew! what a discussion.. if this pew thing is not that crucial, would it be possible to drop them altogether along with virtualenv and let pipenv, venv, and python go hand in hand, as they all recommend one another?

Dependency management/virtual environment solutions for python is too much of a wild terrain right now, and some sacrifices need to be made, imho.

This could scare many people who are not yet ready to quit using virtualenv, but it could be planned for one of the upcoming major releases (if technically feasible).. Change is inevitable, it is just a matter of time.

[FranklinYu]

@doganmeh We can abandon virtualenv altogether, building an equivalent product to only support Python 3.3 or above, but I guess it would be easier to make that another project.

[mkleehammer]

Just a note:

The workaround above mostly works, but pipenv run ... fails for me because there is no activate_this.py under .venv/bin. If issue 21496 is to be believed, there is no plan on adding this file to venv. So even if pew fixes this, pipenv would likely still have a problem.

Is that really necessary? Once you've created a virtual environment with venv, you do not need to activate it to use it. You simply run using the python binary from the venv/bin directory.

[dmtucker]

FWIW, this is related (and what brought me here): http://matplotlib.org/faq/osx_framework.html

In a virtualenv, a non-framework build is used even when the environment is created from a framework build (virtualenv bug #54, virtualenv bug #609).

The solution is to not use virtualenv, but instead the stdlib's venv, which provides similar functionality but without exhibiting this issue.

[omenos]

Dropping Python 2.7/virtualenv support to me doesn’t sound like a particularly terrible idea, if it does allow pipenv to rely on the stdlib of a Python version whose support is not about to expire. As long as it would not corrupt any other part of the pipenv toolchain that’s not easily fixable for the maintainers. Maybe even move 2.7 support to a separate branch/package?

I would like to be able to contribute actual code, but I don’t think I have the experience necessary to do so or the time :frown: The best I can do is add to the discussion at the moment.

[gvoysey]

adding my voice here; i just hit this because of the same issue as @dmtucker .

I am currently resorting to ignoring pipenv completely for this project when on my mac, and making the environment by hand with venv for now.

[JoDoX]

+1 to use venv.
virtualenv have many wontfix bugs.
pipenv completely usefull on Windows 10 with Python 3.7 from Windows Store (virtualenv bug#1362)

[gaborbernat]

virtualenv is moving to venv model - see https://github.com/pypa/virtualenv/issues/1366, that will generate pyvenv.cfg and all the good things, so this should be solved via that.

[encrypromaniac]

---------- Forwarded message ---------
From: Dan Ryan [e-mail]
Date: Mon, Nov 26, 2018, 2:39 PM
Subject: Re: [pypa/pipenv] Use venv instead of virtualenv if available (#15)
To: pypa/pipenv [e-mail]
Cc: Subscribed [e-mail]

Seems right, I saw tox has a workaround, so maybe that would work if
someone ported it over

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pypa/pipenv/issues/15#issuecomment-441533872, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AqyQvrs-RNNTb1YhxaDCWDLpRxrZLKoxks5uy4ydgaJpZM4Lqk2f
.