Pipenv: Provide persistent skip-lock option

[hmaarrfk]

I'm always frustrated when pipenv wants to lock things. This happens when I install and uninstall things. The problem is that it takes FOREVER to lock big scientific packages (that probably include sample data).

I would like to be able to specify completely skipping the locking process within the Pipfile itself.

I've tried to specify --skip-lock in many commands that support it, but some commands simply don't support it. Additionally, if I forget to specify --skip-lock then I have to force quit the program and hope that it maintains a usable state.

[uranusjr]

If this is done it should be a user preference (like the one discussed in #2197), not a project one. Maybe we can implement a PIPENV_SKIP_LOCK or PIPENV_NO_LOCK environ. With that said, I’m not sure why you’d even bother using Pipenv if you don’t lock at all—Pipfile essentially become requirements.txt in this situation.

[hmaarrfk]

Pipfile essentially become requirements.txt in this situation.

Pipenv gives you the virtual environment management. Quite nice IMO.

[hmaarrfk]

Would it be appropriate to place the option within the lockfile itself?
That way it wouldn't be in the Pipfile, but we could still have it on a per-project basis?

[techalchemy]

I think if anything we would want an environment variable for this. We've said no in the past, but I think we've heard it raised enough that we could revisit. I wonder about the security concerns, any thoughts there @ncoghlan? If not I'll run it by Kenneth and we can make a judgment call

[hmaarrfk]

If it makes any difference, I would vote against an environment variable. Having new users to linux/windows set environment variables is rather difficult.

I really think the choice should be on a per project basis for the two reasons:

1. Advanced users: work on one project that requires locked dependencies and an other that is in heavy development where the exact version doesn't matter so much until later in the development cycle.
2. New users: Already have a hard time understanding what python environments are. Since the locking mechanism takes so long on large packages (this doesn't seem like it will change even with caching), it can take 10s of minutes to install something new you find on github. I would prefer asking them to run a command such as: pipenv --three --unlock install which would create a lock file that simply has a "unlocked" flag.

[techalchemy]

Which packages are taking 10s of minutes? If you look at npm, composer, etc, this type of feature is never specified at the project level and there are a lot of good reasons for this — locking is an important element of the behavior we are providing, so disabling it _should_ be an advanced feature, if we choose to offer that at all.

You’re right to consider new users. Adding extra knobs on the command line interface is the best way to confuse them.

[hmaarrfk]

Ok, here is 5 minutes, not 10s, which is an eternity to add when trying to teach people to add new packages to implement new features. I also have an i7 for what it is worth.

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
spyder = "*"
scikit-image = "*"
"pyqt5" = "*"
cython = "*"
matplotlib = "*"
numba = "*"
vispy = "*"

[dev-packages]

[requires]
python_version = "3.6"

$ pipenv --three
$ pipenv install --skip-lock
$ time pipenv lock
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (7d13c5)!
174.69user 18.86system 5:14.28elapsed 61%CPU (0avgtext+0avgdata 687236maxresident)k
2961256inputs+1890392outputs (0major+4989898minor)pagefaults 0swaps

[hmaarrfk]

And here is one from my development environment (15:20.04 elapsed)

$ time pipenv lock
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (fd1cef)!
175.93user 25.78system 15:20.04elapsed 21%CPU (0avgtext+0avgdata 706728maxresident)k
676824inputs+4291400outputs (0major+4954121minor)pagefaults 0swaps
$ cat Pipfile
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
numpy = "*"
matplotlib = "*"
scipy = "*"
imageio = "*"
cython = "*"
wheel = "*"
pip = "*"
numpydoc = "*"
vispy = {git = "https://github.com/vispy/vispy.git"}
"pyqt5" = "*"
jupyter = "*"

[dev-packages]
"pyqt5" = "*"
jupyter = "*"
ipython = "*"
spyder = "*"
pyyaml = "*"
"pep8" = "*"
setuptools = "*"
wheel = "*"
pytest = "*"
sip = "*"

[requires]
python_version = "3.6"

[ncoghlan]

Skipping locking generally isn't a major security concern - OWASP A9 only comes up when you lock your dependencies once, and then never update them (hence why I think figuring out a better way of assessing lock file freshness will let us make --selective-upgrade the default when installing new dependencies).

So I think if folks running a project want to opt out of dependency pinning, and are happy to accept the resulting time-dependent variation in the way different people have their environments configured, that's an entirely reasonable way to go - it's pretty similar to the situation where folks skip checking Pipfile.lock into source control, but goes a step further in not even locking down the local environment.

[kennethreitz42]

skip lock can be an environment variable, but not a pipfile feature.

[uranusjr]

Title edited to reflect resolution.

[devxpy]

With that said, I’m not sure why you’d even bother using Pipenv if you don’t lock at all

I present another argument for enabling such a feature:

The "locking" part of pipenv slows me down drastically while in development.

There are times when I just want to work, and not have pipenv take enormous amounts of cpu, internet and time to just install something.

I will obviously, lock later when I'm in mood :)

PIPENV_SKIP_LOCK env var would be awesome!

[joaqo]

Just want to chime in and validate that this is very useful to several people. Makes pipenv install much faster, which is really cool on simple personal projects. And we still can create the lock whenever we please.

[techalchemy]

I think if we have the prefix for automatically seeking out environment variables enabled this may work by default, but I’m not totally sure. @uranusjr, any idea?

[uranusjr]

Yeah the Click auto prefix thing will probably just work. Is it in the current release yet?

[pauloromeira]

+1 for having that option inside Pipenv.

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
skip_lock = true
...

It could have both options because some projects doesn't need locking and you don't usually track .env files. Other than that, env vars are only useful when you already have isolated environments outside pipenv, otherwise is just simpler to pass skip-lock as an argument.