Pipenv: Consider supporting pip 8+ hash checking mode

[pmac]

Having pipenv lock record hashes for each package version would ensure cryptographically repeatable installation. If this tool isn't meant for use in deployment, then this can be safely ignored, but the Pipfile and Pipfile.lock standards seem to me to be a good start in this direction, and some tooling around this would be nice. I use hashin for this currently, but it has some shortcomings (namely that it doesn't seek out dependencies for you).

Thanks!

[kennethreitz42]

much wanted feature!

[kennethreitz42]

Not sure how to go about implementing

[pmac]

Not sure how to go about implementing

I assumed as much, but didn't see it listed anywhere, so I figured I'd file for safe keeping. I'll think on it as well.

[erikrose]

I'm happy to render at least design aid, since I wrote the hash-checking stuff in pip 8.

[kennethreitz42]

https://github.com/kennethreitz/pipenv/tree/hashes

[kennethreitz42]

i've got a preliminary implementation going, only problem is that it looks like we'd have to write out a temporary requirements file in order for this to work, i think (or use the pip API).

[kennethreitz42]

see https://github.com/kennethreitz/pipenv/pull/72 for details

[erikrose]

Regarding having to write out a temporary reqs file, that's true for now. My thought, though, was always that the reqs-file hash format would ultimately be usable on the commandline as well: pip install smoo==1.2.3 --hash=sha256:2cf24dba5.... We'd have to do some plumbing in pip to support this, making it possible for the order of args to matter. I can't promise to spend time on that in the forseeable future, but I mention it in case anyone else wants to take a swing.

[kennethreitz42]

v3.0.0 released, which includes hashes in the lockfile output, but doesn't use them for installation yet.

[kennethreitz42]

I did it!

v3.1.0 released, which checks hashes upon installation.

[kennethreitz42]

So the process for locking is now as follows:

• run pip download for each dependency.
• analyze the file names for version information, and run pip hash against them to get the hash.

This both speeds up locking, and avoids mucking up the environment during locking.

In addition, $ pipenv install now does the following:

• supports the {hash= TOML syntax
• writes out the entire installation to a temporary file, which pip then installs with --require-hashes enabled.

[kennethreitz42]

If there is an exception (e.g. a hash doesn't match), the output is displayed on the screen during installation.

[pmac]

Wow that was fast. Thanks @kennethreitz!

[erikrose]

Yay. We should all be checking hashes when we deploy. :-) Btw, the version-from-filename thing should get you through 99% of cases. There were only a few packages where it broke.

[pmac]

I notice the Pipfile.lock spec, which you're using correctly it seems, does not make room for multiple hashes per package. The pip hash checking mode does allow for this due to PyPI allowing for multiple downloadable package types. Is the reason that this new system requires platform independent wheels, or has it just not yet come up? The hashin tool I mentioned in comment 0 downloads all the available files for a release (optionally only the ones for a specific Python version) and hashes them all. I am just curious if this is a case that's being considered. Thanks again!