Pipenv: Simplified Vendoring

[jar349]

Is your feature request related to a problem? Please describe.

The problem is an increase in the frequency of software supply-chain attacks (example).

Describe the solution you'd like

I'd like to use pipenv to vendor the wheels for the packages in my Pipfile.lock into a folder in my app (to be checked in).

I'd like to use pipenv to install wheels from the vendor folder (during the CI pipeline and after deployment)

Describe alternatives you've considered

A private pypi server that houses "blessed" packages. I do not prefer this approach due to the overhead of maintaining a blessed list of packages and also of the private server itself.

Additional context

I do not have an opinion on how such a thing could be implemented, but I have an idea in my head of what I imagine. Something like:

$ pipenv --python 3.7
$ pipenv install --vendor-dir=./vendor requests
$ git add .
$ git commit -m "adds requests to the app"
$ git push

so now I've vendored requests.

$ pipenv sync --vendor-dir=./vendor

and now I've installed from the vendor folder

I don't know. Perhaps vendoring package-by-package is too complicated (and maybe defeats the purpose of vendoring). Maybe it should be an all or nothing:

$ pipenv vendor --dir=./vendor on

just straight-up changes the [[source]] in Pipfile to the ./vendor dir

Don't know. You all would know better.

[uranusjr]

I feel this is out of the scope of Pipenv and should be implemented in its own tool instead.

[jar349]

I feel this is out of the scope of Pipenv and should be implemented in its own tool instead.

I feel that vendoring is squarely within the packaging lifecycle and that pipenv's goal is to be the one-stop shop for python packaging. Introducing a new tool would take us back to the days when we were using two programs (pip and virtualenv) for our packaging needs.

However, if it were its own tool (let's call it pipvend) then pipenv would - at a minimum - be asked to install packages from a vendored directory.

$ pipvend init
created wheelhouse in ./vendor
Pipfile updated to use wheelhouse as package source

$ pipvend install requests
downloaded requests-1.2.3.whl to wheelhouse
Pipfile updated
Pipfile.lock updated

$ pipenv sync  
# notices that the Pipefile source points to a local directory 
# and installs from the .whl files there

In this scenario, is pipenv already able to install from a local directory?

[jar349]

There's a lot of pipenv code that would be re-written in a separate pipvend tool:

• resolving a name to a pypa package
• retrieving and validating packages
• resolving and retrieving dependencies
• Updating Pipfile and Pipfile.lock

Probably more. This is the hint that makes me believe that vendoring belongs inside pipenv.

$ pipenv --python 3.7 --vendor ./vendor
created wheelhouse in ./vendor

$ pipenv install  --vendor requests
# updates pipfile, downloads .whl to wheelhouse, installs package into venv, relocks

# check it all in
--
# deploy app somewhere
$ pipenv sync --vendor # installs from `./vendor` subfolder 

☝️ seems like the right thing.

[nix7drummer88]

FWIW I agree with @jar349 here, being able to vendor within pipenv would be excellent, and I'm surprised this isn't a feature already.

I'm currently looking to vendor packages for one of my projects that uses pipenv, and it's looking like the only way to do it at present is to go back to pip, which forces to re-evaluate our choice of using pipenv at all.

[costincaraivan]

I'd have to add that for some newfangled serverless stuff such as AWS Lambda, there is no support for pipenv.

They only support regular pip for dependencies, and they basically use that through vendoring.

They run pip to download the dependencies and then those get bundled into dependency "layers" which are then used by the serverless function.

So far, so bad.

Now, with pipenv, it gets even worse. pipenv is totally unusable with AWS Lambda, from what I see, because there is no obvious way to vendor dependencies. I'll probably have to revert to pip (it sucks, but in this case I'll have to choose Lambda over pipenv 🤷‍♂️).

[fridex]

Now, with pipenv, it gets even worse. pipenv is totally unusable with AWS Lambda, from what I see, because there is no obvious way to vendor dependencies. I'll probably have to revert to pip (it sucks, but in this case I'll have to choose Lambda over pipenv ).

Maybe micropipenv can be helpful there?!