Pipenv: SyntaxError: invalid syntax

[pglombardo]

When running pipenv I'm getting a syntax error. python -m pipenv.help also results in a Syntax error.

This might be due to pipenv although I'm not completely convinced. Any direction, hints or ideas would be appreciated

Python 3.5.2 (via pyenv)
pip 9.0.1 from /Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages (python 3.5)
pipenv (11.8.0) installed via pip
MacOS Sierra

---

Expected result

No syntax errors

Actual result

Stacktrace output:

$ python -m pipenv.help                                                                                                                                                     [17:06:06]
Traceback (most recent call last):
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/help.py", line 9, in <module>
    from .core import project, system_which, find_python_in_path, python_version
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/core.py", line 24, in <module>
    import pipdeptree
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/pipdeptree.py", line 16, in <module>
    import pip9
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/pip9/__init__.py", line 28, in <module>
    from pip9.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/pip9/vcs/subversion.py", line 9, in <module>
    from pip9.index import Link
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/pip9/index.py", line 31, in <module>
    from pip9.wheel import Wheel, wheel_ext
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/pip9/wheel.py", line 6, in <module>
    import compileall
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/compileall.py", line 20, in <module>
    from concurrent.futures import ProcessPoolExecutor
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/../concurrent/futures/__init__.py", line 8, in <module>
    from concurrent.futures._base import (FIRST_COMPLETED,
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/../concurrent/futures/_base.py", line 414
    raise exception_type, self._exception, self._traceback
                        ^
SyntaxError: invalid syntax
FAIL: 1


$ pipenv                                                                                                                                                                    [17:15:16]
Traceback (most recent call last):
  File "/Users/pglombardo/.pyenv/versions/3.5.2/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==11.8.0', 'console_scripts', 'pipenv')()
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 696, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 621, in make_context
    self.parse_args(ctx, args)
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 1018, in parse_args
    rest = Command.parse_args(self, ctx, args)
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 875, in parse_args
    parser = self.make_parser(ctx)
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 821, in make_parser
    for param in self.get_params(ctx):
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/click/core.py", line 774, in get_params
    help_option = self.get_help_option(ctx)
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/cli.py", line 26, in get_help_option
    from .import core
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/core.py", line 23, in <module>
    import pipdeptree
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/pipdeptree.py", line 16, in <module>
    import pip9
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/pip9/__init__.py", line 28, in <module>
    from pip9.vcs import git, mercurial, subversion, bazaar  # noqa
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/pip9/vcs/subversion.py", line 9, in <module>
    from pip9.index import Link
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/pip9/index.py", line 31, in <module>
    from pip9.wheel import Wheel, wheel_ext
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/vendor/pip9/wheel.py", line 6, in <module>
    import compileall
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/compileall.py", line 20, in <module>
    from concurrent.futures import ProcessPoolExecutor
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/../concurrent/futures/__init__.py", line 8, in <module>
    from concurrent.futures._base import (FIRST_COMPLETED,
  File "/Users/pglombardo/.pyenv/versions/3.5.2/lib/python3.5/site-packages/pipenv/../concurrent/futures/_base.py", line 414
    raise exception_type, self._exception, self._traceback
                        ^
SyntaxError: invalid syntax
FAIL: 1

Steps to replicate

run any command with pipenv

[pglombardo]

Ugly hack-around is to modify that line to only raise the exception:

 #raise exception_type, self._exception, self._traceback
raise self._exception

[damouse]

Same issue on osx with v11.8.0, but on python 3.6.2.

Also the same error I got when I made issue #1589 last week.

@pglombardo's fix works if you just need to get moving.

EDIT: see below for actual fix.

[riebschlager]

FWIW, I'm getting this same error when I deploy my python application to Heroku.

Update: I was able to get Heroku builds working again by changing my runtime.txt from python-3.6.4 to python-3.6.2, commit and push and then change it back again.

[damouse]

The issue seems to arise from having futures installed in python 3 via pip, which is meant to backport functionality already baked into recent versions of python3. This is not the same as #1738, which has futures installed in 2.7-- this one is a little more obvious.

You can tell if you have the problem if you see that the exception is being thrown from python3.X/site-packages/concurrent or pip3 freeze contains:

futures==3.1.1

The fix is to uninstall the package:

pip3 uninstall futures

Verifying that importing concurrent.futures resolves to the system version:

~ $ python3
Python 3.6.2 (default, Aug 28 2017, 16:38:53) 
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import concurrent.futures
>>> concurrent.futures.__file__
'/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/concurrent/futures/__init__.py'

No more issues with pipenv after this.

[uranusjr]

@damouse Making sure before I close this: so this is a setup problem, correct? Pipenv does not install futures, and there is nothing more we can do here if the user has a broken environment.

[damouse]

@uranusjr yeah I think you're clear, its not pipenv's fault

[georgebrock]

I was seeing this issue on Heroku too, and had to clear the slug build cache to solve it:

heroku plugins:install heroku-repo
heroku repo:purge_cache

(I agree this should be closed, just leaving this here for other people who encounter this problem and find this issue while searching for a resolution.)

[KeynesYouDigIt]

Clever pickup @damouse ! @uranusjr I just got this error, is there a way to warn for it? would running pipenv clean clear it up? i directly uninstalled futures and its getting stuck looking for that package when I run install again

[uranusjr]

@KeynesYouDigIt You might want to make sure you’re running a recent version of Pipenv though. I believe the situation has been greately improved in 11.8.3. You might not need to uninstall futures if the changes work for you.

[KeynesYouDigIt]

Thanks @uranusjr - A full upgrade of pip and pipenv got it moving

[ssbarnea]

As stated ^ I encountered this issue with pipenv, version 2018.11.26 and latest pip. Uninstalling futures fixed it but it would be better to find a way to avoid it.

[uranusjr]

The easiest “fix” is to convince maintainers of futures to switch to a more compatible syntax.

[ssbarnea]

@uranusjr I think is not as easy as it appears because effectively futures project says that it supposed to be used only on py2. Check https://pypi.org/project/futures/3.3.0/#files -- But the 3.1.1 release is the last one that does not mention this on its wheel.

This means that any project that lists futures as a requirement without adding the extra bits for installing it only on py2 will break us.

I am sure that futures will say is not their problem. I wonder how can I find who is installing futures on py3....

[ssbarnea]

This thread should give more insights https://github.com/agronholm/pythonfutures/issues/83

[uranusjr]

Yeah I was afraid that would be the case as well. While I sympathise the technical argument (it really is not their fault), at some point package maintainers need to realise when downstream dependants mess things up you still take the “blame”, and it makes things easier to be practical in certain situations.

But I digress. Pipenv could do better to prevent a rogue package from breaking the world as well, but unfortunately it is quite a degree more difficult to implement, and we lack the resources to do it (especially when it is obvious how much less resource it’d take to fix it elsewhere). The problem is as structural (to OSS) as technical.