Pipenv: Allow Pipfile to consume environment variables

[dvf]

It would be awesome if Pipfile could be parsed to allow for environment variables.

While it's great that pipenv supports private indexes, it's annoying if those indexes require credentials—it's bad practice to store them in source control:

[[source]]

url = "https://{{ MY_USERNAME }}:{{ MY_PASSWORD }}@my_custom_repo.jfrog.io/simple"
verify_ssl = true
name = "pypi"

Happy to contribute a PR for this if this gets some thumbs.

[kennethreitz42]

:+1:

[gsemet]

@kennethreitz what do you think about making this section able to inherits values from a global setting file, such as /etc/pipenv.conf (a-la /etc/pip.conf), so all Pipfile within a private organisation can use the same internal repository (nexus, artifactory, other,...)

[kennethreitz42]

no

[techalchemy]

@gsemet I believe pipfiles will use sources specified in your pip.conf now though

[kennethreitz42]

When we create a pipfile for you, it will be populated with sources from pip.conf.

[gsemet]

not sure, we use our internal nexus cache in pip.conf:

$ cat /etc/pip.conf
[global]
trusted-host = nexus.mycompagny.fr
index = http://nexus.mycompagny.fr/repository/pypi/pypi
index-url = http://nexus.mycompagny.fr/repository/pypi/simple

when executing pip install on its own, it uses this server.

when creating a pipfile:

$ pipenv install request
...
$ cat Pipfile                                                                                                                                                                                              
[[source]]
name = "pypi"
verify_ssl = true
url = "https://pypi.python.org/simple"
...

[kennethreitz42]

we pick up user pip conf i think, not global pip conf.

[kennethreitz42]

here's the code: https://github.com/pypa/pipenv/blob/master/pipenv/project.py#L419

[dvf]

That sounds potentially dangerous if you’re storing credentials in pip.conf.

@kennethreitz, I was reading another issue where you referred to the Pipfile as the source of truth. I think this is a great idea and should stay that way.

On Mar 14, 2018, at 9:14 AM, Kenneth Reitz [e-mail] wrote:

here's the code: https://github.com/pypa/pipenv/blob/master/pipenv/project.py#L419

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[kennethreitz42]

we could detect high-entropy strings automatically (or just urlparse).

[kennethreitz42]

@dvf pipfile is def the source of truth.

[kennethreitz42]

i think environment variable injection is a good idea though

[kennethreitz42]

url = "https://$PYPI_USERNAME:[email protected]/simple"

[kennethreitz42]

or just https://[email protected]/simple

[kennethreitz42]

We raise an error if PYPI_CREDS isn't in os.environ.

[dvf]

If we only check PYPI_CREDS, we're assuming that a user would only have a single source defined in their Pipfile. I think we should allow for any environment variable—similar to how Docker allows you to use Environment Variables in Compose files.

• Pipfile is the ultimate source of truth
• Any value can contain an environment variable, but nothing else

[kennethreitz42]

Of course we'd support any.

[kennethreitz42]

i'm just using that as an example

[dvf]

Oh gotcha 👍

Do you have any ideas about how you see this being implemented?

[jtratner]

@kennethreitz - env variable expansion is something I'd be happy to implement unless somebody else wants to (prob have time next weekend), but also happy to leave to other ambitious and/or helpful folks

[dvf]

@jtratner I was going to take a stab at it. Feel free to leave some suggestions in advance though.

[jtratner]

oh go for it @dvf :)

[techalchemy]

Resolved by #1769 and #1809

[njgrisafi]

It would be nice if we can have environment variables loaded throughout the entire Pipfile. E.g.

[dev-packages]
my-package = {path="${PATH_TO_REPO}", editable = true}

[andyneff]

@njgrisafi The reason this does not work appears to be due to https://github.com/pypa/pipenv/blob/8aa9283fda2e8cc532f517969f5f3e1969e8d4a2/pipenv/project.py#L519 using tomlkit instead of pipfile.api.PipfileParser

However, this gets further complicated by the fact that the path is copied into the Pipfile.lock. The way inject currently works, it that it injects the values at parse time, which means by the time you get to writing the lock file, it will write the value of the the environment variable, not the ${FOO} syntax. :(

I agree, this feature working on path would be very much desired for me too. I want to run a Pipfile in and out of a docker container, but currently implementation limits this.