Pipenv: Installation of non-required dependency

[dfee]

One of my dependencies is using ruamel.yaml, which has a requirement on ruamel.ordereddict if the installation is happening on CPython <= 2.7.

This dependency does not install on Python 3, and it shouldn't be installed – something pip honors.

Actually, pipenv honors this too, for an initial installation. However, it adds ruamel.ordereddict to the Pipfile.lock anyway, and the wheels come off there when doing a fresh installation of the pipenv. I.e.:

pipenv --rm
pipenv install --three  # now attempts to install ruamel.ordereddict, which has c bindings that won't compile.

Maybe I'm throwing edge cases at you today. But can we figure this out?

Here's the lockfile:

{
    "_meta": {
        "hash": {
            "sha256": "0078ef581d9a09fffae8a4d729aaa6130bcbd078ecb6fb6be73221a9104b41f1"
        },
        "host-environment-markers": {
            "implementation_name": "cpython",
            "implementation_version": "3.6.1",
            "os_name": "posix",
            "platform_machine": "x86_64",
            "platform_python_implementation": "CPython",
            "platform_release": "16.7.0",
            "platform_system": "Darwin",
            "platform_version": "Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64",
            "python_full_version": "3.6.1",
            "python_version": "3.6",
            "sys_platform": "darwin"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "ruamel.ordereddict": {
            "hashes": [
                "sha256:150ce8e6c514a2a2b62753622a75874962561f8e5eeec81a3172ab952807bf0b",
                "sha256:45541836cbfdde630033cae7bbbe35acbac87a0ceec79f944b7a3bedd940fe78",
                "sha256:aee2fa23e884249b4284b728888c553d551e5bfd4de2731f10153fd7813ec55f",
                "sha256:08b4b19fe518d32251a5338e039c4dc9eb0876f2919f94c9b8d2f9446ea80806",
                "sha256:bf0a198c8ce5d973c24e5dba12d3abc254996788ca6ad8448eabc6aa710db149"
            ],
            "version": "==0.4.13"
        },
        "ruamel.yaml": {
            "hashes": [
                "sha256:4bdec0b1a4a12e4a35b788ef53e0b2efd1e9f815ca48615b5244ccdbc8f0b56b",
                "sha256:6432088ef1c3cdc4b304dcec6044e560160c1f51dd0360ae3f0887057d310056",
                "sha256:64868964b64cad9103f37d0b4fc0ec950e5b683bbc47bf4a1ea893fd70aaabf0",
                "sha256:c8da0ea47df5f2dee58f6f904335ac55098b7e72303b01414ce80f37dd3091df",
                "sha256:87bdfb04981f1040ec60b1d1bfa8e7e4d89f7d06c86d4b2fa935002c62e43380",
                "sha256:c20e853cb585167fd5e4ce6e1b57fefe7bd4a61659f8fec5f77e12efa59a7a77",
                "sha256:e884979944642b4ad3222cb102a0d4c17fed2d37abfffdf0f319fae6e41d436a",
                "sha256:10493c92c0e5bd57d576d73b708ed900532846ead1b381c5236c41d0c39f5f71",
                "sha256:3127a87b274e516b3bf2058f64d784089322fddd7322f8e78e9f5380bdfee064",
                "sha256:530bec874292bacbbb80a9143e5182ce436c2a2434a2ea83dda24d30b8e572df",
                "sha256:d92d90c9bc0945223e47223a67808dd97ac9390ed914cc6871479b7ba489e607",
                "sha256:3e87114aac9553c39eea4b88e258eb7fdce39b81a2c399a775efe1f64e4f3d7b",
                "sha256:dfd14829070728159d0dac55a19e4c77514cb8ad3ae3fc2ed065d7f24038b60f",
                "sha256:2f28a3b6665697c20d841a4aee21cfb932bb0db91f293ff97daf845b914ddddb",
                "sha256:2de7bd2d5713c46be9d1d489b028178c9497813f78bd0486a31bbe1c268d7f21",
                "sha256:ddf0e1277664fafff0ae692e8ea2fca56f1ee4daf686d9be785ccdf3a9542744",
                "sha256:f1e29054c6e477963e302b007b6cd1d6c7a58c38d78fabe64fde9ce170d2d1fd"
            ],
            "version": "==0.15.34"
        }
    },
    "develop": {}
}

And, here's the output of pip freeze:

$ pipenv run pip freeze
ruamel.yaml==0.15.34

[kennethreitz42]

this has been biting me too  — for maya.

[dfee]

I just noticed maya shares that dep. I guess the temporary workaround would be to clear ruamel.ordereddict from the lock file.

[kennethreitz42]

I dont' think we can do anything about this — a proper setup.py file would specify ramuel.ordereddict; python_version > 3, but they're doing something crazy complicated, that i don't think we can get around.

[kennethreitz42]

fixed this!

[kennethreitz42]

I'm amazing.

[kennethreitz42]

https://github.com/kennethreitz/pipenv/commit/05933ae9fdca722e8bf49b617f37b5053a40b825

[kennethreitz42]

v8.0.1 fixes this. :)

[kennethreitz42]

💫✨🍰✨💫

[dfee]

Hmm. I'm still having this problem. Can you verify that pipenv --three install ruamel.yaml doesn't actually add ruamel.ordereddict to the lockfile?

/tmp/hello
$ pipenv --version
pipenv, version 8.0.1

[dfee]

Oh boy. For some reason this is working in a docker container I created specifically for pipenv, but not on my Mac.

[dfee]

Could there be a cache issue on my mac?

[dfee]

Ah ha, so I think I can replicate the issue:

Using python3 (works!):

python3.6 -m venv env
source env/bin/activate
pip install pipenv
deactivate
env/bin/pipenv install --three ruamel.yaml

... and we get a proper lockfile (no ruamel.ordereddict).

Using python2 (the problem!):

virtualenv env
source env/bin/activate
pip install pipenv
deactivate
env/bin/pipenv install --three ruamel.yaml

... and we get an improper lockefile (containst ruamel.ordereddict)

This reflects why my docker image (with only Python3 installed) has no issues, and my Mac, with a pipsi install of pipenv against Python2.7 fails.

[kennethreitz42]

you have to use the proper version of python when generating the lockfile.

[kennethreitz42]

that's why the lockfile says requires_python

[kennethreitz42]

--three is not doing anything for you if you're running in a virtualenv. run pipenv outside of a virtualenv.

[dfee]

So what are we supposed to make of the "fancy install process" that uses pipsi? Is that not a good approach?

I did end up installing pipsi with Python3, and that overcame my problem. But I guess --two would fail now.

[kennethreitz42]

pipsi does not run pipenv in a virtualenv. it just installs it in one.

[TimSimpsonR]

@dfee I was just having this same problem and wanted to comment that your solution works.

For all the Googlers out there if you're on OSX check this:

~/.local/venvs/pipenv/bin/python --version
~/.local/venvs/pipsi/bin/python --version

If it doesn't say Python 3, do this:

pipsi uninstall pipenv
pipsi uninstall pipsi
curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python3
pipsi install pipenv