Pipenv: Unhelpful message if local package build (setup.py egg_info) fails with underlying exception

[greysteil]

All of the below is using master, checked out at 0fa7bd8bd9be0c64dd14d6c8421b617651d3745d.

Running pipenv lock for a Pipfile that imports a setup.py with a setup_requires dependency fails with a TypeError: __init__() got an unexpected keyword argument 'download_dir' error.

$ python -m pipenv.help output

Pipenv version: '2018.05.18'

Pipenv location: '/Users/greysteil/code/pipenv/pipenv'

Python location: '/usr/local/opt/python/bin/python3.6'

Other Python installations in PATH:

• 2.6: /usr/bin/python2.6
• 2.6: /usr/bin/python2.6
• 2.7: /usr/bin/python2.7
• 2.7: /usr/bin/python2.7
• 3.6: /usr/local/bin/python3.6m
• 3.6: /usr/local/bin/python3.6

• 3.6: /usr/local/bin/python3.6

• 3.6.5: /usr/local/bin/python

• 3.6.5: /usr/local/bin/python
• 2.7.10: /usr/bin/python
• 3.6.5: /usr/local/bin/python3
• 3.6.5: /usr/local/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '16.7.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 16.7.0: Tue Jan 30 11:27:06 PST '
                     '2018; root:xnu-3789.73.11~1/RELEASE_X86_64',
 'python_full_version': '3.6.5',
 'python_version': '3.6',
 'sys_platform': 'darwin'}

System environment variables:

• TERM_PROGRAM
• PYENV_ROOT
• TERM
• SHELL
• CLICOLOR
• TMPDIR
• Apple_PubSub_Socket_Render
• TERM_PROGRAM_VERSION
• TERM_SESSION_ID
• USER
• SSH_AUTH_SOCK
• __CF_USER_TEXT_ENCODING
• LSCOLORS
• PATH
• PWD
• EDITOR
• LANG
• XPC_FLAGS
• RBENV_SHELL
• XPC_SERVICE_NAME
• HOME
• SHLVL
• LOGNAME
• OLDPWD
• _
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /Users/greysteil/.pyenv/bin:/Users/greysteil/.cargo/bin:/usr/local/heroku/bin:/Users/greysteil/.rbenv/shims:/usr/local/bin:./node_modules/.bin:.bundle/binstubs:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/git/bin:/Library/TeX/texbin:/usr/local/sbin
• SHELL: /bin/bash
• EDITOR: subl -w
• LANG: en_GB.UTF-8
• PWD: /Users/greysteil/code/python-test

---

Contents of Pipfile ('/Users/greysteil/code/python-test/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]
"e1839a8" = {path = ".", editable = true}

---

Expected result

Install should succeed

Actual result

Locking [dev-packages] dependencies…
Using pip: -i https://pypi.org/simple

                          ROUND 1                           
Current constraints:
  file:///Users/greysteil/code/python-test (from -r /var/folders/kv/y02nj9zd5ss16sn505tk6g_40000gn/T/pipenv-hs7ym9sw-requirements/pipenv-0igg_jkv-constraints.txt (line 2))

Finding the best candidates:
  found candidate -e file:///Users/greysteil/code/python-test (constraint was <any>)

Finding secondary dependencies:

INFO:notpip._internal.operations.prepare:Obtaining file:///Users/greysteil/code/python-test (from -r /var/folders/kv/y02nj9zd5ss16sn505tk6g_40000gn/T/pipenv-hs7ym9sw-requirements/pipenv-0igg_jkv-constraints.txt (line 2))
Traceback (most recent call last):
  File "/Users/greysteil/code/pipenv/pipenv/patched/piptools/repositories/pypi.py", line 305, in get_legacy_dependencies
    wheel_cache=self.wheel_cache,
TypeError: __init__() got an unexpected keyword argument 'download_dir'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/greysteil/code/pipenv/pipenv/resolver.py", line 84, in <module>
    main()
  File "/Users/greysteil/code/pipenv/pipenv/resolver.py", line 73, in main
    system=system,
  File "/Users/greysteil/code/pipenv/pipenv/resolver.py", line 64, in resolve
    allow_global=system,
  File "/Users/greysteil/code/pipenv/pipenv/utils.py", line 403, in resolve_deps
    req_dir=req_dir,
  File "/Users/greysteil/code/pipenv/pipenv/utils.py", line 295, in actually_resolve_deps
    resolved_tree.update(resolver.resolve(max_rounds=PIPENV_MAX_ROUNDS))
  File "/Users/greysteil/code/pipenv/pipenv/patched/piptools/resolver.py", line 104, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/Users/greysteil/code/pipenv/pipenv/patched/piptools/resolver.py", line 204, in _resolve_one_round
    for dep in self._iter_dependencies(best_match):
  File "/Users/greysteil/code/pipenv/pipenv/patched/piptools/resolver.py", line 278, in _iter_dependencies
    for dependency in self.repository.get_dependencies(ireq):
  File "/Users/greysteil/code/pipenv/pipenv/patched/piptools/repositories/pypi.py", line 246, in get_dependencies
    legacy_results = self.get_legacy_dependencies(ireq)
  File "/Users/greysteil/code/pipenv/pipenv/patched/piptools/repositories/pypi.py", line 339, in get_legacy_dependencies
    self.resolver.resolve(reqset)
  File "/Users/greysteil/code/pipenv/pipenv/patched/notpip/_internal/resolve.py", line 107, in resolve
    self._resolve_one(requirement_set, req)
  File "/Users/greysteil/code/pipenv/pipenv/patched/notpip/_internal/resolve.py", line 264, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/Users/greysteil/code/pipenv/pipenv/patched/notpip/_internal/resolve.py", line 198, in _get_abstract_dist_for
    req, self.require_hashes, self.use_user_site, self.finder,
  File "/Users/greysteil/code/pipenv/pipenv/patched/notpip/_internal/operations/prepare.py", line 354, in prepare_editable_requirement
    abstract_dist.prep_for_dist(finder, self.build_isolation)
  File "/Users/greysteil/code/pipenv/pipenv/patched/notpip/_internal/operations/prepare.py", line 155, in prep_for_dist
    self.req.run_egg_info()
  File "/Users/greysteil/code/pipenv/pipenv/patched/notpip/_internal/req/req_install.py", line 486, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/Users/greysteil/code/pipenv/pipenv/patched/notpip/_internal/utils/misc.py", line 698, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pipenv.patched.notpip._internal.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /Users/greysteil/code/python-test/

Steps to replicate

Run pipenv lock in a directory with the following:

# Pipfile
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]
"e1839a8" = {path = ".", editable = true}

# setup.py
import setuptools

def _main():
    setuptools.setup(
        packages=setuptools.find_packages(),
        setup_requires=['pbr'],
        pbr=True)


if __name__ == "__main__":
    _main()

[techalchemy]

I think #2314 handles this— can you check?

[greysteil]

Just checked - sadly not 😢.

[uranusjr]

I did some digging. The exception you see is masking the real error, which can be obtained if you run python setup.py egg_info directly:

λ .venv\Scripts\python.exe setup.py egg_info
Traceback (most recent call last):
  File "setup.py", line 12, in <module>
    _main()
  File "setup.py", line 8, in _main
    pbr=True)
  File "C:\Users\uranusjr\Documents\playground\pbr-test\.venv\lib\site-packages\setuptools\__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "C:\Users\uranusjr\AppData\Local\Programs\Python\Python36\Lib\distutils\core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "C:\Users\uranusjr\Documents\playground\pbr-test\.venv\lib\site-packages\setuptools\dist.py", line 370, in __init__
    k: v for k, v in attrs.items()
  File "C:\Users\uranusjr\AppData\Local\Programs\Python\Python36\Lib\distutils\dist.py", line 281, in __init__
    self.finalize_options()
  File "C:\Users\uranusjr\Documents\playground\pbr-test\.venv\lib\site-packages\setuptools\dist.py", line 529, in finalize_options
    ep.load()(self, ep.name, value)
  File "c:\users\uranusjr\documents\playground\pbr-test\.eggs\pbr-4.0.4-py3.6.egg\pbr\core.py", line 92, in pbr
    'The setup.cfg file %s does not exist.' % path)
distutils.errors.DistutilsFileError: The setup.cfg file C:\Users\uranusjr\Documents\playground\pbr-test\setup.cfg does not exist.

PBR requires a setup.cfg containing needed information, but you’re not providing it.

We need to somehow get Pipenv to display the underlying error instead of a generic “Command "python setup.py egg_info" failed with error code 1”. It is really unhelpful.

[MarkLodato]

Is there a workaround when one gets this error message? I am new to pipenv and virtualenv so I am not sure what to do next. Can I use an older version of pipenv or pip? Run some command manually?

[MarkLodato]

Nevermind. In my case, the root cause was #2584.

[techalchemy]

Thanks for the info, that may be a secret culprit on a lot of related issues now that you mention it...

[techalchemy]

This will be reported better in the next release

Using the example above:

 !  /t/test  pipenv install -e .
Creating a virtualenv for this project…
Pipfile: /tmp/test/Pipfile
Using /home/hawk/.pyenv/versions/3.7.0/bin/python3.7 (3.7.0) to create virtualenv…
⠴Already using interpreter /home/hawk/.pyenv/versions/3.7.0/bin/python3.7
Using base prefix '/home/hawk/.pyenv/versions/3.7.0'
/home/hawk/.pyenv/versions/3.7.0/lib/python3.7/site-packages/virtualenv.py:1041: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
New python executable in /home/hawk/.virtualenvs/test-hxkKlP5o/bin/python3.7
Also creating executable in /home/hawk/.virtualenvs/test-hxkKlP5o/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /home/hawk/.virtualenvs/test-hxkKlP5o
Creating a Pipfile for this project…
Installing -e .…
Obtaining file:///tmp/test
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/test/setup.py", line 12, in <module>
        _main()
      File "/tmp/test/setup.py", line 8, in _main
        pbr=True)
      File "/home/hawk/.virtualenvs/test-hxkKlP5o/lib/python3.7/site-packages/setuptools/__init__.py", line 140, in setup
        return distutils.core.setup(**attrs)
      File "/home/hawk/.pyenv/versions/3.7.0/lib/python3.7/distutils/core.py", line 108, in setup
        _setup_distribution = dist = klass(attrs)
      File "/home/hawk/.virtualenvs/test-hxkKlP5o/lib/python3.7/site-packages/setuptools/dist.py", line 370, in __init__
        k: v for k, v in attrs.items()
      File "/home/hawk/.pyenv/versions/3.7.0/lib/python3.7/distutils/dist.py", line 292, in __init__
        self.finalize_options()
      File "/home/hawk/.virtualenvs/test-hxkKlP5o/lib/python3.7/site-packages/setuptools/dist.py", line 529, in finalize_options
        ep.load()(self, ep.name, value)
      File "/tmp/test/.eggs/pbr-4.2.0-py3.7.egg/pbr/core.py", line 92, in pbr
        'The setup.cfg file %s does not exist.' % path)
    distutils.errors.DistutilsFileError: The setup.cfg file /tmp/test/setup.cfg does not exist.

    ----------------------------------------

Error:  An error occurred while installing -e .!
Command "python setup.py egg_info" failed with error code 1 in /tmp/test/

This is likely caused by a bug in -e .. Report this to its maintainers.

[astrojuanlu]

I'm still getting unhelpful errors when there is an installation failure (in my case, a new cftime release was failing because they didn't upload a wheel and I was missing Cython). Any way to enable the debugging?

[techalchemy]

--verbose will likely help

[nickwilliams-eventbrite]

--verbose does not help. I still see only this and no other error information:

pipenv.patched.notpip._internal.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/tmpjmd83hjxbuild/pyproj/

I don't have any issues installing that library with pip.

[sewagodimo]

I am getting the same error as @nickwilliams-eventbrite
I also don't get the error when using pip

[alexykot]

I'm facing a similar problem with installing pysqlite via pipenv install. If I try to pip install pysqlite, I get the following error:

[alexykot@alexykot-T580 project]$ 
Collecting pysqlite
  Using cached https://files.pythonhosted.org/packages/42/02/981b6703e3c83c5b25a829c6e77aad059f9481b0bbacb47e6e8ca12bd731/pysqlite-2.8.3.tar.gz
    Complete output from command python setup.py egg_info:
    pysqlite is not supported on Python 3. When using Python 3, use the sqlite3 module from the standard library.

    ----------------------------------------

This makes sense and is a meaningful response.

However when I run pipenv install while having pysqlite = "*" in the [packages] section - I'm getting a long stack trace and meaningless

...
pipenv.patched.notpip._internal.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/tmpplvb860ubuild/pysqlite/``` 

The original error is no longer there, and this is not helpful. Seems like pipenv is swallowing the original response and conceals useful messages.

[frostming]

can u try the master branch and see what it prints?