Virtualenv: RuntimeError: 'Exec format error' when using pipenv+pyenv-installed python 3.8.1 to install environment

[mhoonjeon]

Thanks for submitting an issue!

If submitting a BUG please provide:

• [X ] Minimal reproducible example or detailed descriptions

/Users/mhjeon/.local/share/virtualenvs/nowwow-0D8RInBf/bin/python3 -mvirtualenv /tmp/env -p /Users/mhjeon/.local/share/virtualenvs/nowwow-0D8RInBf/bin/python3

This gives me an output of

RuntimeError: failed to query /Users/mhjeon/.pyenv/versions/3.8.1/Python.framework/Versions/3.8/python with code 8 err: 'Exec format error'

• [ ] OS and pip list output
  using virtualenv 20.0.0b2

Please see this issue for more detail

[gaborbernat]

What's the content of the executable in question? Says it's not executable 🤔 Can you post please the output by passing the -vvv flag? Please also attach the ls -al of the executable in question. Thanks!

[gaborbernat]

Please also attach the command you've used to install your pyenv python in question 😊

[gaborbernat]

This issue still needs a small reproducible script, I'm failing to do so. A run with -vvv would be a good start.

[smitt04]

A reproducible script for me was

mkdir -p /tmp/vtest && cd /tmp/vtest
virtualenv --python python3.7 py37
. ./py37/bin/activate
pip install virtualenv
./py37/bin/python -m virtualenv --no-download --python ./py37/bin/python py37

and I get the following output:

RuntimeError: failed to query /opt/local/Library/Frameworks/Python.framework/Versions/3.7/python with code 8 err: 'Exec format error'

[gaborbernat]

Can you please post the output by passing -vvv to the virtualenv command.

[smitt04]

➜ ./py37/bin/python -m virtualenv -vvv --no-download --python ./py37/bin/python py37
118 setup logging to NOTSET [DEBUG report:43]
128 find interpreter for spec PythonSpec(path=/private/tmp/vtest/py37/bin/python) [INFO builtin:44]
128 discover system for PythonInfo(spec=CPython3.7.4.final.0-64, exe=/private/tmp/vtest/py37/bin/python, platform=darwin, version='3.7.4 (default, Oct 16 2019, 11:14:51) \n[Clang 11.0.0 (clang-1100.0.33.8)]', encoding_fs_io=utf-8-UTF-8) in /opt/local/Library/Frameworks/Python.framework/Versions/3.7 [DEBUG py_info:332]
129 filesystem is not case-sensitive [DEBUG info:28]
130 Attempting to acquire lock 4446500432 on /Users/ksmithson/Library/Application Support/virtualenv/py-info/20.0.1/c405059ba5cab83b4c9edb814600b939aab4bb299cf178dce99ee642f825d112.lock [DEBUG filelock:270]
130 Lock 4446500432 acquired on /Users/ksmithson/Library/Application Support/virtualenv/py-info/20.0.1/c405059ba5cab83b4c9edb814600b939aab4bb299cf178dce99ee642f825d112.lock [INFO filelock:274]
130 get interpreter info via cmd: /opt/local/Library/Frameworks/Python.framework/Versions/3.7/python -s /private/tmp/vtest/py37/lib/python3.7/site-packages/virtualenv/discovery/py_info.py [DEBUG cached_py_info:107]
132 Attempting to release lock 4446500432 on /Users/ksmithson/Library/Application Support/virtualenv/py-info/20.0.1/c405059ba5cab83b4c9edb814600b939aab4bb299cf178dce99ee642f825d112.lock [DEBUG filelock:315]
132 Lock 4446500432 released on /Users/ksmithson/Library/Application Support/virtualenv/py-info/20.0.1/c405059ba5cab83b4c9edb814600b939aab4bb299cf178dce99ee642f825d112.lock [INFO filelock:318]
132 done in 129ms [INFO __main__:23]
132 RuntimeError: failed to query /opt/local/Library/Frameworks/Python.framework/Versions/3.7/python with code 8 err: 'Exec format error' [ERROR __main__:30]

[asottile]

I'm a little sad we still released while we had identified this during the beta period -- would have been nice to work this out instead of having users pester downstream packages about this https://github.com/pre-commit/pre-commit/issues/1327

[asottile]

here's what I'm getting:

$ virtualenv -vvv vvv -p python2.7
69 setup logging to NOTSET [DEBUG report:43]
75 find interpreter for spec PythonSpec(implementation=CPython, major=2, minor=7) [INFO builtin:44]
75 discover system for PythonInfo(spec=CPython3.7.2.final.0-64, exe=/private/tmp/venv/bin/python3, platform=darwin, version='3.7.2 (v3.7.2:9a3ffc0492, Dec 24 2018, 02:44:43) \n[Clang 6.0 (clang-600.0.57)]', encoding_fs_io=utf-8-UTF-8) in /Library/Frameworks/Python.framework/Versions/3.7 [DEBUG py_info:332]
77 filesystem is not case-sensitive [DEBUG info:28]
79 Attempting to acquire lock 4549774472 on /Users/asottile/Library/Application Support/virtualenv/py-info/20.0.1/fb7ec9939e66e8ed189e2fe772cae61fcc992510c23be22bd7dadeab5367beb4.lock [DEBUG filelock:270]
79 Lock 4549774472 acquired on /Users/asottile/Library/Application Support/virtualenv/py-info/20.0.1/fb7ec9939e66e8ed189e2fe772cae61fcc992510c23be22bd7dadeab5367beb4.lock [INFO filelock:274]
79 get interpreter info via cmd: /Library/Frameworks/Python.framework/Versions/3.7/python -s /private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py [DEBUG cached_py_info:107]
81 Attempting to release lock 4549774472 on /Users/asottile/Library/Application Support/virtualenv/py-info/20.0.1/fb7ec9939e66e8ed189e2fe772cae61fcc992510c23be22bd7dadeab5367beb4.lock [DEBUG filelock:315]
82 Lock 4549774472 released on /Users/asottile/Library/Application Support/virtualenv/py-info/20.0.1/fb7ec9939e66e8ed189e2fe772cae61fcc992510c23be22bd7dadeab5367beb4.lock [INFO filelock:318]
82 done in 80ms [INFO __main__:26]
82 RuntimeError: failed to query /Library/Frameworks/Python.framework/Versions/3.7/python with code 8 err: 'Exec format error' [ERROR __main__:33]
Traceback (most recent call last):
  File "/private/tmp/venv/bin/virtualenv", line 8, in <module>
    sys.exit(run_with_catch())
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/__main__.py", line 31, in run_with_catch
    run(args)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/__main__.py", line 19, in run
    run_via_cli(args)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/run/__init__.py", line 21, in run_via_cli
    session = session_via_cli(args)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/run/__init__.py", line 28, in session_via_cli
    parser = build_parser(args)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/run/__init__.py", line 41, in build_parser
    parser._interpreter = interpreter = discover.interpreter
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/discover.py", line 44, in interpreter
    self._interpreter = self.run()
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/builtin.py", line 33, in run
    return get_interpreter(self.python_spec)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/builtin.py", line 46, in get_interpreter
    for interpreter, impl_must_match in propose_interpreters(spec):
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/builtin.py", line 63, in propose_interpreters
    yield PythonInfo.current_system(), True
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 268, in current_system
    cls._current_system = cls.from_exe(sys.executable, raise_on_error=True, resolve_to_host=True)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 290, in from_exe
    proposed = proposed._resolve_to_system(proposed)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 318, in _resolve_to_system
    target = target.discover_exe(prefix=prefix, exact=False)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 341, in discover_exe
    info = self.from_exe(exe_path, resolve_to_host=False)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/py_info.py", line 287, in from_exe
    proposed = from_exe(cls, exe, raise_on_error=raise_on_error, ignore_cache=ignore_cache)
  File "/private/tmp/venv/lib/python3.7/site-packages/virtualenv/discovery/cached_py_info.py", line 37, in from_exe
    raise result
RuntimeError: failed to query /Library/Frameworks/Python.framework/Versions/3.7/python with code 8 err: 'Exec format error'

I had to adjust __main__.py to raise instead of sys.exit(1) to get the stacktrace

$ diff -u venv/lib/python3.7/site-packages/virtualenv/__main__.py{.old,}
--- venv/lib/python3.7/site-packages/virtualenv/__main__.py.old 2020-02-10 15:43:05.000000000 -0800
+++ venv/lib/python3.7/site-packages/virtualenv/__main__.py 2020-02-10 15:41:28.000000000 -0800
@@ -28,6 +28,7 @@
         run(args)
     except (KeyboardInterrupt, Exception) as exception:
         logging.error("%s: %s", type(exception).__name__, exception)
+        raise
         sys.exit(1)

[asottile]

note that I am not using pyenv, my python is acquired from homebrew

[gaborbernat]

@asottile did not need the stack trace, knew exactly where things fail, but needed to see the discovery trace to see where things go wrong. Thanks.

[gaborbernat]

@asottile how did you install virtualenv?

[asottile]

the same as @smitt04's directions above

[gaborbernat]

Hello, a fix for this issue has been released via virtualenv 20.0.2; see https://pypi.org/project/virtualenv/20.0.2/ (https://virtualenv.pypa.io/en/latest/changelog.html#v20-0-2-2020-02-11) . Please give a try and report back if your issue has not been addressed; if not, please comment here, and we'll reopen the ticket. We want to apologize for the inconvenience this has caused you and say thanks for having patience while we resolve the unexpected bugs with this new major release.

[asottile]

I get past the error now, but now I'm hitting #1583 from the same directions