Pipenv: `pip install` from Pypi does not install dependencies

[gsemet]

Hello

I really prefer pipenv to handle Pipfile and Pipfile.lock over the old requirements.txt, however, my packages that uses pipenv (ex: here), when installed with pip install txrwlock, get no dependency at all.

I wonder if, for the moment at least, auto-generating a requirements.txt is not necessary, at least waiting for official support of Pipfile by pip (and other tools such as ReadTheDoc or Pyup that only support requirements.txt).

What's your thoughts?

[vphilippon]

I do not see a need to auto-generate requirements.txt, that could actually cause some conflict with an existing requirements.txt in the directory and such.

We can generate a requirements.txt with pipenv lock --requirements > requirements.txt, so the option is there for anyone needing a requirements.txt for system on which they can't install Pipenv, until pip supports Pipfile/Pipfile.lock.

[gsemet]

I agree, this should be done on purpose (no auto generation). For my issue, so every modules pushed to Pypi should have this requirements.txt automatically done during build before building sdist/bdist/wheel ? Because for the moment, I don't have the dependencies of my package installed when I do pip install mypackage

The issue with pipenv lock --requirements > requirements.txt is that is freeze ALL dependencies (default and develop). I have written a little tool to keep 2 requirements files (requirements.txt and requirements-dev.txt) from the Pipfile: pipenv-to-requirements

[vphilippon]

First:

Because for the moment, I don't have the dependencies of my package installed when I do pip install mypackage

I'm not sure I follow what you're expecting here, maybe I'm missing something.
I've read the description of pipenv-to-requirements, and I have the feeling you're aware of the following, but I'll but it there anyway:
Even if a requirements.txt was shipped with your package, doing pip install mypackage would not read anything from the requirements.txt: it's still all part of the abstract dependencies defined in the setup.py (and txrwlock, for instance, defines none). You would still have to do pip install -r requirements.txt. Of course, with Pipfile, nothing of that changes: it doesn't define any dependencies of the package itself. Its a replacement to the requirements.txt
Like I said, if you were aware of all this, what did I miss in your question?

Second:

The issue with pipenv lock --requirements > requirements.txt is that is freeze ALL dependencies (default and develop).

Ah, yes, that's discussed in https://github.com/kennethreitz/pipenv/issues/942.
It might not be that easy though to deal with though.

Finally:

I have written a little tool to keep 2 requirements files (requirements.txt and requirements-dev.txt) from the Pipfile

Personal idea: You could use that tool to obtain those 2 separate requirements file from the Pipfile, and then use pip-tools to perform the dependency resolution (lock) on the requirements.txt alone (at the moment pipenv uses a patched version of pip-tools to perform dependency resolution itself) before shipping it.
That's a bit of work, its not perfect, but that's a workaround if you want to start working and promoting the use of Pipfile while still working with other systems.

[gsemet]

I use PBR that does all of this automatically. But it does not support Pipfile yet, apparently.

[gsemet]

Weirdly, I get the following with requirements.txt and Pipfile both aligned:

• pip install txrwlock does install future on Python 2.6
• pipenv install txrwlock does not

future is under markers in the dependency Pipfile and requirements.txt. So, there must be something related to conditional installation (markers) with setup.py dependency declaration (here, done automatically by PBR)

[techalchemy]

Hmmm @vphilippon haven't we seen this bug before? Doesn't this depend on how _pipenv_ is installed and whether it will respect environment markers? The details are a bit murky at the moment but I'm pretty sure this might relate to #857

[erinxocon]

I should note that we do not officially support python 2.6. Not that it's causing this, just wanted to acknowledge it.

[erinxocon]

@Stibbons can you post an automatically generated setup.py from pbr? I'm not familiar with PBR.

[gsemet]

I have fixed this by ensuring i don’t use markers in my library. If i resume:

• a lib A uses à markers such as python_version < 3.0
• both requirements.txt and Pipfile have the same content with their respective syntax
• setup.py référence the dependencies (I use PBR to do it, i have not tested with find_packages, but in both cases I don’t see how the markers can be described in it)
• symptom is: pip install mypackage does install the conditioned dependency on python 2.7, pipenv install mypackage does not

[gsemet]

PBR does not generate a setup.py, it is pluged into it and does everything « magically ». But it definitely support « requirements.txt », it is described on their doc, I have opened an issue on their side to support Pipfile as well.

By the way, parsing à Pipfile using pipenv adds so many dependencies (ex: flake8), is it possible to split the parsing part from the pipenv CLI in two different packages?). That could help other projects to read Pipfile

[vphilippon]

@techalchemy I think you're right, especially if I take @Stibbons comment:

symptom is: pip install mypackage does install the conditioned dependency on python 2.7, pipenv install mypackage does not

This might be yet another case of "the python version running pipenv itself is impacting the actual dependency resolution".
@Stibbons Could you tell us which python version you used globally to install pipenv itself?
Bonus question: are you using pyenv?

And finally

By the way, parsing à Pipfile using pipenv adds so many dependencies (ex: flake8), is it possible to split the parsing part from the pipenv CLI in two different packages?). That could help other projects to read Pipfile

That should be discussed in a separate issue. Tracking becomes impossible otherwise 😄 .

[gsemet]

Donc in #967 :)

I don't use pyenv, only brew python on my mac and default python 2.7 on ubuntu 16.04. I have seen this behavior on both the Travis build and my local machine.
I also have some instabilities of pipenv on Python 2.7 (see the various executions in here), but if you tell me it is not officially supported that would be a totally acceptable response. I am not sure this instability on the Travis build is due to pipenv of by travis it self (bad infra?)