Pipenv: Support a target directory for vendoring (similar to pip --target)

[benjaminjkraft]

We currently vendor some of our dependencies to a target directory, using pip install -t. It would be great for pipenv to support this functionality! This is our main blocker to further use.

I think right now it may be possible to accomplish by having pipenv generate a requirements.txt and pass it to pip to do the install, but that's very manual, and loses features like integrity-checking, so it would be much nicer for pipenv to do it.

(I mentioned this at PyCon, just wanted to make a github issue too so I can follow it.)

[uranusjr]

You can use this as a work around:

$ pipenv lock -r > /somewhere-temporary/requirements.txt   # Use -d if you want dev-packages as well.
$ pipenv run pip install -t -r /somewhere-temporary/requirements.txt

[uranusjr]

A couple of possible API designs:

• pipenv lock --packages=/vendor-target
• pipenv vendor /vendor-target

Both would essentially just produce a requirements.txt (maybe in-memory), and use pip install -t to install them into the target directory. This is not very out of place in Pipenv (given we already have lock -r), but still requires additional approval.

In the mean time, feel free to create your own tool for this, and share it here.

[brettcannon]

Can the path argument as used for editable dependencies be used but without specifying editable (Otherwise what is the path argument for)? If it can I realize that's a bit tedious to specify for every argument, but it would be an option if path can be contorted like that.

[techalchemy]

@brettcannon I believe it technically could be, although it would be breaking. I don't think the --target option would be that hard to implement, and I believe I actually talked to Kenneth about this already and we both thought it was ok. Because of the constraints on pip install etc, path dependencies need to point at an installable filesystem path (i.e. with a setup.py or a location that is an artifact or a wheel).

[mcintyre94]

Just to add to this, in my issue (#2445) I proposed the best API for me would be:

Describe the solution you'd like
pipenv install -t path/to/src installs a dependency into Pipfile/Pipfile.lock, and runs pip install -t path/to/src .

I think it'd make sense not to do the equivalent of pipenv install at the same time, ie it'd be like only running pip -t and match its behaviour. Just the dependency would end up managed by pipenv.

Note that pip does't have a pip uninstall -t, I think it'd be okay not to have a pipenv uninstall -t also (though pipenv uninstall should still remove it from Pipfile).

--

This would make pipenv really nice for managing AWS lambdas, where you need to bundle your dependencies with the source to deploy, but you might want to do a normal pipenv install of the dependencies to run tests (outside of the deployed directory).

[kennethreitz42]

we are tracking this in our private tracker.

[scoates]

Is there a way that people without access to the private tracker are able to find out about the status of this kind of thing? I got here by searching for exactly this feature…

[macrovve]

Any update of this feature?

[pirate]

Any update on this :heart:? Really wish we could specify a manual .venv output path so we can use pipenv in our docker builds:

WORKDIR /app
COPY ./Pipfile /app/
COPY ./Pipfile.lock /app/
RUN env PIPENV_SKIP_LOCK=1 \
        PIPENV_VENV_IN_PROJECT=1 \
        PIPENV_IGNORE_VIRTUALENVS=1 \
        pipenv install --clear --skip-lock --target=/app/.docker-venv
ENV PATH="/app/.docker-venv:${PATH}"
...

This way our entire /app code directory can be mounted as a docker volume shared with the host, without the host's /app/.venvconflicting with the venv inside the container ( /app/.docker-venv). They need to be separate venvs because the host might be a different OS entirely (E.g. macOS host, alpine container).

The equivalent feature in other package managers is widely used and appreciated, e.g. npm --prefix ., cargo --root ., pip --target, I don't see why this was rejected for pipenv.

[maikelrobier]

@kennethreitz @techalchemy was this idea dropped? Was there a conclusion on this issue?

[scoates]

Now that there's new Pipenv momentum, and some of the leadership appears to have changed, is there any way we could reopen this issue? As far as we can tell from the comments here, this was never resolved, and was somehow opaquely moved to a secret tracker.

At the very least, could we track this in public, please?

[scoates]

3+ weeks and not even a reply. I don't think the request was unreasonable. Was hoping for more with the new wave of Pipenv interest. )-:

[istrupin]

I would also like this functionality -- in my case for keeping packages in a directory for upload into AWS Lambda.

AWS suggests installing dependencies into the local project folder here: https://docs.aws.amazon.com/lambda/latest/dg/python-package.html

The --target flag would really help here.

[istrupin]

You can use this as a work around:

$ pipenv lock -r > /somewhere-temporary/requirements.txt   # Use -d if you want dev-packages as well.
$ pipenv run pip install -t -r /somewhere-temporary/requirements.txt

This doesn't actually work for me -- I get an error saying: ERROR: Invalid requirement: 'package/requirements.txt' Hint: It looks like a path. It does exist.

What you need to do is:

$ pipenv lock -r > /somewhere-temporary/requirements.txt   # Use -d if you want dev-packages as well.
$ pipenv run pip install -t /path/to/install/location -r /somewhere-temporary/requirements.txt