Pipenv: Expand environment variables in path field

[andyneff]

It would be nice if the injection of environment variables worked for the path variable too

@njgrisafi from #1688

It would be nice if we can have environment variables loaded throughout the entire Pipfile. E.g.

[dev-packages]
my-package = {path="${PATH_TO_REPO}", editable = true}

Is your feature request related to a problem? Please describe.

Environment expansion does not work for path or file as observed in #2317

Describe the solution you'd like

I'd like the variable expansion to work for the path/file fields. Unfortunately, this probably involves an expansion in the Pipfile.lock too, so that both can truly be portable with respect to the environment variable

Describe alternatives you've considered

I tried this in pipenv/project.py as a failed POC.

    def _parse_pipfile(self, contents):
        from pipfile.api import PipfileParser

        try: 
            d = tomlkit.parse(contents)
        except Exception:
            # We lose comments here, but it's for the best.)
            # Fallback to toml parser, for large files.
            d = toml.loads(contents)
        return PipfileParser().inject_environment_variables(d)

I will have to use symlinks in the meantime

[heron182]

Would be nice to have this feature.
Currently we need to install an #egg from a private gitlab repo and the ability to inject the user token would be nice, such as:

mypkg = {git = "https://oauth2:[email protected]/my-repo#egg=mypkg"}

[frostming]

This seems not true regarding #3751

[northtree]

@frostming, https://pipenv.readthedocs.io/en/latest/advanced/#support-for-environment-variables

Pipenv supports the usage of environment variables in place of authentication fragments in your Pipfile. These will only be parsed if they are present in the [[source]] section.

[northtree]

@andyneff It seems that PipfileParser was only used for testing? I couldn't find other code to import PipfileParser (and call inject_environment_variables).

➜  pipenv git:(master) rg PipfileParser        
tests/unit/test_vendor.py
13:from pipfile.api import PipfileParser
16:class TestPipfileParser:
20:        p = PipfileParser()

pipenv/patched/pipfile/api.py
59:class PipfileParser(object):
71:        return '<PipfileParser path={0!r}'.format(self.filename)
159:        p = PipfileParser(filename=filename)

[bkemmer]

I am having the same problem trying to insert GitLab token on the URL.
I tried the snippet proposed by the documentation, but it doesn't work

Pipfile

[[source]]
url = "https://${GIT_EMAIL}:${GIT_ACCESS_TOKEN}@<private_git_repo>/foo/bar.git"
verify_ssl = true
name = "repo_name"

[packages]
lib_name = {editable = true, index = "repo_name", ref = "v1.0"}

tried changing the key index to git without success too, receiving the same error:

ERROR:

  File "/Users/.../.local/share/virtualenvs/hash-lWDy_qFk/lib/python3.7/site-packages/pipenv/vendor/requirementslib/models/utils.py", line 592, in validate_path
    raise ValueError("Invalid path {0!r}".format(value))
ValueError: Invalid path 'repo_name'

[frostming]

@bkemmer Your Pipfile is not valid, [[source]] is for PyPI index, not VCS repo.

[maxsei]

bit of a workaround but I managed to do environment variable substitution with this:
sh -c 'cp Pipfile Pipfile_; read -p "password: " PASSWORD; set -a; envsubst < Pipfile_ > Pipfile; pipenv lock; mv Pipfile_ Pipfile'

check https://github.com/a8m/envsubst for details on installing the tool