Pipenv: Installation of non-required dependency with matplotlib

[grangerp]

I think the problem is related to issue #701 but version 8.0.2 did not resolve the problem.

How to reproduct:

$ pipenv install matplotlib --python 3.6
Creating a virtualenv for this project…
Using /usr/bin/python3.6m to create virtualenv…
⠋Running virtualenv with interpreter /usr/bin/python3.6m
Using base prefix '/usr'
New python executable in /home/pgranger/.virtualenvs/tmat-V-42yd7s/bin/python3.6m
Also creating executable in /home/pgranger/.virtualenvs/tmat-V-42yd7s/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/pgranger/.virtualenvs/tmat-V-42yd7s
Creating a Pipfile for this project…
Installing matplotlib…
Collecting matplotlib
  Using cached matplotlib-2.0.2-cp36-cp36m-manylinux1_x86_64.whl
Collecting six>=1.10 (from matplotlib)
  Using cached six-1.11.0-py2.py3-none-any.whl
Collecting numpy>=1.7.1 (from matplotlib)
  Using cached numpy-1.13.1-cp36-cp36m-manylinux1_x86_64.whl
Collecting cycler>=0.10 (from matplotlib)
  Using cached cycler-0.10.0-py2.py3-none-any.whl
Collecting pytz (from matplotlib)
  Using cached pytz-2017.2-py2.py3-none-any.whl
Collecting pyparsing!=2.0.0,!=2.0.4,!=2.1.2,!=2.1.6,>=1.5.6 (from matplotlib)
  Using cached pyparsing-2.2.0-py2.py3-none-any.whl
Collecting python-dateutil (from matplotlib)
  Using cached python_dateutil-2.6.1-py2.py3-none-any.whl
Installing collected packages: six, numpy, cycler, pytz, pyparsing, python-dateutil, matplotlib
Successfully installed cycler-0.10.0 matplotlib-2.0.2 numpy-1.13.1 pyparsing-2.2.0 python-dateutil-2.6.1 pytz-2017.2 six-1.11.0

Adding matplotlib to Pipfile's [packages]…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock!

Pipfile:

[[source]]                                                                                          
url = "https://pypi.python.org/simple"                                                              
verify_ssl = true                                                                                   
name = "pypi"                                                                                       

[dev-packages]                                                                                      

[packages]                                                                                          
matplotlib = "*"                                                                                    

[requires]                                                                                          
python_version = "3.6" 

Pipfile.lock:

{
    "_meta": {
        "hash": {
            "sha256": "761e94a47a13b0512242b245c0758cb75e331caeb8a84355b5e98b0a04781379"
        },
        "host-environment-markers": {
            "implementation_name": "cpython",
            "implementation_version": "3.6.1",
            "os_name": "posix",
            "platform_machine": "x86_64",
            "platform_python_implementation": "CPython",
            "platform_release": "4.10.0-35-generic",
            "platform_system": "Linux",
            "platform_version": "#39-Ubuntu SMP Wed Sep 13 07:46:59 UTC 2017",
            "python_full_version": "3.6.1",
            "python_version": "3.6",
            "sys_platform": "linux"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.6"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.python.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "cycler": {
            "hashes": [
                "sha256:1d8a5ae1ff6c5cf9b93e8811e581232ad8920aeec647c37316ceac982b08cb2d",
                "sha256:cd7b2d1018258d7247a71425e9f26463dfb444d411c39569972f4ce586b0c9d8"
            ],
            "version": "==0.10.0"
        },
        "functools32": {
            "hashes": [
                "sha256:f6253dfbe0538ad2e387bd8fdfd9293c925d63553f5813c4e587745416501e6d",
                "sha256:89d824aa6c358c421a234d7f9ee0bd75933a67c29588ce50aaa3acdf4d403fa0"
            ],
            "version": "==3.2.3.post2"
        },
        "matplotlib": {
            "hashes": [
                "sha256:30a3d8700dcad90eb43dcbde5107ccc5ec67241f34de377de11dbef371e4a683",
                "sha256:c23aca1fb3546b4b60a8ff16c1318874da05bf0dc47accd9cd818a99ad7b35d2",
                "sha256:025f49079f840dc887e593d86aa0cc02671bd9363919f209c1145fc12743df94",
                "sha256:8168cc902aaa965f033c099238b80f5a8853664324c5837b10fee688c12ec3b6",
                "sha256:d0da5ceba1a77130603b7b8cc3e5b8c58b7e733d3948fcec1fcf4aa812b024cc",
                "sha256:2a251c6c87cceb4677e3c35a8e1c73973d05025d8ed6322861102e1a336db5f8",
                "sha256:8814f728b1f037ebc20e7216f0c498bdd17daefa1a44d99f2466a4c21d0ff541",
                "sha256:1da49090f6286f54cdbccc423bed35417f621172d9aecbfc9608b87b99a73a11",
                "sha256:8e009ec991f39dc1c0f6f891bfb68e794a08ac98b5c343274099da2865e2f932",
                "sha256:afd8cb8194ee5500d7b5b9be034d9c13b0a3db0288d8988abaaa94b4ec7bbca5",
                "sha256:481da65b536183497952d0f620defea88206edb8387839a2802951956492cb1d",
                "sha256:64410b788c81e1f50ad7066d245e0a3421fbc278d2e26c34e2e4681f2a023b64",
                "sha256:c7bae47ab1c15da0265f8aa2d183457afd05c78fbb77e960a346bc87b3fdb57c",
                "sha256:ec9f09fd35624b735271997527e2c2fd7f905d1612ce5dabdda4836a15ab7411",
                "sha256:647c02a9f333cb7055f42527cf2bd85fb3615fd8de0cce4648d327efaffe5458",
                "sha256:7671819e29e43635f4e1fddc06dd86cd9d5a2fdeb67312538791e222e7ae3af7",
                "sha256:64dcf5e99d6c0b100903d4c8a339dbac6e0822870ef420fc09f656deb1dd73c6",
                "sha256:0ffbc44faa34a8b1704bc108c451ecf87988f900ef7ce757b8e2e84383121ff1"
            ],
            "version": "==2.0.2"
        },
        "numpy": {
            "hashes": [
                "sha256:91a4f5c6594a61b57b0ab6031a084fa3686b1e847cc2215983e444583594b529",
                "sha256:ab6abc2083013dd86a8fcba2ba16bab00690cb81db62588781d656572809c9a9",
                "sha256:02e6279d95081086469e6ed83c708c4c48ed03a28ab87c71bea28af3b95fa56d",
                "sha256:05a7a81397e1391ae34cc0d14764a31ab6f73dbd0abe0952b3550d3ad4df265d",
                "sha256:73fd54d9787f4f8747f823a7e2d0693da94c66b670ccf436e4bb488bbcd5ce8c",
                "sha256:4b7da62ba159bfc5fee6f54709b0708686ee15081f16dc5f81cda7f1e0e77941",
                "sha256:1980c4bc1eb495624c8414f3763da83b91d37c3c69772ab6912e9a857a143cdb",
                "sha256:436d47018c3cd2b9723ed3cd4ed4698ea7641449c71096781478ef6a20ae3bd0",
                "sha256:43722270fbfb07d91558985a3da37aa92a2d4e2d271182526959a5773f9fb12a",
                "sha256:838e48df3703c8747f355cd6386e0680b906a2f7b2bbd304e8a2d531692484ce",
                "sha256:1400ec59c7f6c4f9390cc3bc5e56a6cbae2c30b39024eef317a0b52fe9c174c6",
                "sha256:ed6a909a78e29a4056e30f918a26b231e33edc77bd785bbceb461877baf9feb5",
                "sha256:c1833829526ce8f5177a3e07554b6c98c194072f66f018839ecd1ef2d15e6c4a",
                "sha256:a09a4707066fe9431c6b79a1be922bc126f4bc50502ae7e9f67d40917d0cc6d4",
                "sha256:9a8515002f143a5934f25ad2aacdfd1fcf57a7f5da6142c439eb8787ef65e8a6",
                "sha256:b49caeb170e54cc59863017a199667a51526bd906bcd5ee340fcf0e01bd7fa94",
                "sha256:405c3dbb6a57415ec8576ff1c0248f332ac1c3be2e5eea04d498dad8431bf57b",
                "sha256:42b3cf886701bb16f3bdf2ae6c39af67b464cdd67d5fc86619ef2a876a23de27",
                "sha256:94cb6ef9ffd15d7d904d0825ada642a51dc8890cdc06f1e4fb8e46cff79fe2ef",
                "sha256:d910a24f536f926bd56fb30d6f17ae8b89a1406e105087a49e014e000b00e8db",
                "sha256:b064211a4d86fc8009ef90c66d1443ba4a0c56d481659e085a190299569955e3",
                "sha256:f4b4b2da8c1b4f7c212742d2be03aa9277d46fd7b309025d930ad554e5739932",
                "sha256:c9b0283776085cb2804efff73e9955ca279ba4edafd58d3ead70b61d209c4fbb"
            ],
            "version": "==1.13.1"
        },
        "pyparsing": {
            "hashes": [
                "sha256:fee43f17a9c4087e7ed1605bd6df994c6173c1e977d7ade7b651292fab2bd010",
                "sha256:0832bcf47acd283788593e7a0f542407bd9550a55a8a8435214a1960e04bcb04",
                "sha256:9e8143a3e15c13713506886badd96ca4b579a87fbdf49e550dbfc057d6cb218e",
                "sha256:281683241b25fe9b80ec9d66017485f6deff1af5cde372469134b56ca8447a07",
                "sha256:b8b3117ed9bdf45e14dcc89345ce638ec7e0e29b2b579fa1ecf32ce45ebac8a5",
                "sha256:8f1e18d3fd36c6795bb7e02a39fd05c611ffc2596c1e0d995d34d67630426c18",
                "sha256:e4d45427c6e20a59bf4f88c639dcc03ce30d193112047f94012102f235853a58"
            ],
            "version": "==2.2.0"
        },
        "python-dateutil": {
            "hashes": [
                "sha256:95511bae634d69bc7329ba55e646499a842bc4ec342ad54a8cdb65645a0aad3c",
                "sha256:891c38b2a02f5bb1be3e4793866c8df49c7d19baabf9c1bad62547e0b4866aca"
            ],
            "version": "==2.6.1"
        },
        "pytz": {
            "hashes": [
                "sha256:c883c2d6670042c7bc1688645cac73dd2b03193d1f7a6847b6154e96890be06d",
                "sha256:03c9962afe00e503e2d96abab4e8998a0f84d4230fa57afe1e0528473698cdd9",
                "sha256:487e7d50710661116325747a9cd1744d3323f8e49748e287bc9e659060ec6bf9",
                "sha256:43f52d4c6a0be301d53ebd867de05e2926c35728b3260157d274635a0a947f1c",
                "sha256:d1d6729c85acea5423671382868627129432fba9a89ecbb248d8d1c7a9f01c67",
                "sha256:54a935085f7bf101f86b2aff75bd9672b435f51c3339db2ff616e66845f2b8f9",
                "sha256:39504670abb5dae77f56f8eb63823937ce727d7cdd0088e6909e6dcac0f89043",
                "sha256:ddc93b6d41cfb81266a27d23a79e13805d4a5521032b512643af8729041a81b4",
                "sha256:f5c056e8f62d45ba8215e5cb8f50dfccb198b4b9fbea8500674f3443e4689589"
            ],
            "version": "==2017.2"
        },
        "six": {
            "hashes": [
                "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb",
                "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9"
            ],
            "version": "==1.11.0"
        },
        "subprocess32": {
            "hashes": [
                "sha256:1e450a4a4c53bf197ad6402c564b9f7a53539385918ef8f12bdf430a61036590",
                "sha256:7e787be117aa7db8fd856998cf27ebafb3fcf3919eee4981d9edb5d22dd2d052"
            ],
            "version": "==3.2.7"
        }
    },
    "develop": {}
}

$ pipenv run pip freeze
cycler==0.10.0
matplotlib==2.0.2
numpy==1.13.1
pyparsing==2.2.0
python-dateutil==2.6.1
pytz==2017.2
six==1.11.0

Everything installed properly.

But when I run:

$ pipenv install
Installing dependencies from Pipfile.lock…
An error occurred while installing subprocess32==3.2.7! Will try again.
An error occurred while installing functools32==3.2.3.post2! Will try again.
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 9/9 — 00:00:01
Installing initially–failed dependencies…
Collecting subprocess32==3.2.7 ▉▉▉▉▉▉ 0/2 — 00:00:00
  Using cached subprocess32-3.2.7.tar.gz
    Complete output from command python setup.py egg_info:
    This backport is for Python 2.x only.

    ----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-jl5xj5ia/subprocess32/

  ☤  ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/2 — 00:00:00

subprocess32 and functools32 should only be installed on python2.

Any idea why v8.0.1 dod not fix this?

Thanks

[kennethreitz42]

I'll take a look. Are you running pipenv within a virtualenv?

[kennethreitz42]

matplotlib has a crazy setup.py

[kennethreitz42]

as a workaround, you could put the following in your pipfile:

subprocess32 = {version="*" python_version="<'3'"}

[grangerp]

pipenv is installed with pipsi, so yes it is in a virtualenv, I will dig if this installation can cause the problem.

Yes I have dig a bit in mapplotlib setup.py and I am not sure to understand everything they do.
Thanks for the workaround.

[kennethreitz42]

no, it's not "running in a virtualenv" in that instance. The environment variables for the virtualenv aren't set.

I reproduced this.

[kennethreitz42]

fixed!

[kennethreitz42]

patching pip is fun

[kennethreitz42]

v8.0.3 is out which fixes this! ✨🍰✨

[gopik]

Installing matplotlib fails with:

% (command_desc, proc.returncode, cwd))

pip.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/tmp5potdw1sbuild/subprocess32/

pipenv, version 8.2.6
python version -Python 3.6.2 :: Anaconda, Inc.

(tried both outside and inside virtualenv).

[AlJohri]

@kennethreitz I still seem to be running into this issue with the latest pipenv.

To replicate, just create a new directory and run the following command: pipenv install --python 3.6 matplotlib

This is the stacktrace for reference:

Creating a virtualenv for this project…
Using /Users/johria/.pyenv/versions/3.6.2/bin/python3.6m to create virtualenv…
⠋Running virtualenv with interpreter /Users/johria/.pyenv/versions/3.6.2/bin/python3.6m
Using base prefix '/Users/johria/.pyenv/versions/3.6.2'
New python executable in /Users/johria/.local/share/virtualenvs/test99-H_BrxzXF/bin/python3.6m
Also creating executable in /Users/johria/.local/share/virtualenvs/test99-H_BrxzXF/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /Users/johria/.local/share/virtualenvs/test99-H_BrxzXF
Creating a Pipfile for this project…
Installing matplotlib…
Collecting matplotlib
  Downloading matplotlib-2.1.0-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (13.2MB)
Collecting cycler>=0.10 (from matplotlib)
  Using cached cycler-0.10.0-py2.py3-none-any.whl
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 (from matplotlib)
  Using cached pyparsing-2.2.0-py2.py3-none-any.whl
Collecting python-dateutil>=2.0 (from matplotlib)
  Using cached python_dateutil-2.6.1-py2.py3-none-any.whl
Collecting numpy>=1.7.1 (from matplotlib)
  Downloading numpy-1.13.3-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (4.5MB)
Collecting six>=1.10 (from matplotlib)
  Using cached six-1.11.0-py2.py3-none-any.whl
Collecting pytz (from matplotlib)
  Using cached pytz-2017.2-py2.py3-none-any.whl
Installing collected packages: six, cycler, pyparsing, python-dateutil, numpy, pytz, matplotlib
Successfully installed cycler-0.10.0 matplotlib-2.1.0 numpy-1.13.3 pyparsing-2.2.0 python-dateutil-2.6.1 pytz-2017.2 six-1.11.0

Adding matplotlib to Pipfile's [packages]…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Traceback (most recent call last):
  File "/Users/johria/.pyenv/versions/3.6.2/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==8.2.7', 'console_scripts', 'pipenv')()
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/cli.py", line 1847, in install
    do_lock(system=system, pre=pre)
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/cli.py", line 1082, in do_lock
    pre=pre
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/utils.py", line 452, in resolve_deps
    resolved_tree.update(resolver.resolve(max_rounds=PIPENV_MAX_ROUNDS))
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 102, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 200, in _resolve_one_round
    for dep in self._iter_dependencies(best_match):
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 296, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 153, in get_dependencies
    result = reqset._prepare_file(self.finder, ireq)
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/patched/pip/req/req_set.py", line 639, in _prepare_file
    abstract_dist.prep_for_dist()
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/patched/pip/req/req_set.py", line 134, in prep_for_dist
    self.req_to_install.run_egg_info()
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/patched/pip/req/req_install.py", line 438, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/Users/johria/.pyenv/versions/3.6.2/lib/python3.6/site-packages/pipenv/patched/pip/utils/__init__.py", line 707, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /var/folders/c5/sxpknfp571v3ydglf4305g9m0000gn/T/tmp5vt4td9bbuild/subprocess32/

$ python --version
3.6.2

$ pyenv versions
  system
* 2.7.13 (set by /Users/johria/.pyenv/version)
* 3.6.2 (set by /Users/johria/.pyenv/version)

$ pipenv --version
pipenv, version 8.2.7

[AlJohri]

Retracting comment above - I seem have have gotten it to work. Unsure what exactly happened above.