Tox: PyPy 3 on Windows fails with "The parameter is incorrect"

[pganssle]

Unfortunately I don't have a Windows machine easily available, so I can't manually reproduce this, but on Github Actions it seems like python -m tox consistently fails on PyPy on Windows. Here is a failing job.

Details, in case the GHA link ever dies:

Run python -m tox
  python -m tox
  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
  env:
    TOXENV: py
    TEST_EXTRAS_TOX: 
    CODECOV_TOKEN: ***
    pythonLocation: C:\hostedtoolcache\windows\PyPy\3.6.9\x86
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\interpreters\__init__.py", line 25, in get_executable
    return self.name2executable[envconfig.envname]
KeyError: '.package'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\lib-python\3\runpy.py", line 196, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\lib-python\3\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\__main__.py", line 4, in <module>
    tox.cmdline()
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\session\__init__.py", line 44, in cmdline
    main(args)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\session\__init__.py", line 69, in main
    exit_code = session.runcommand()
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\session\__init__.py", line 197, in runcommand
    return self.subcommand_test()
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\session\__init__.py", line 213, in subcommand_test
    venv.package = self.hook.tox_package(session=self, venv=venv)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\callers.py", line 208, in _multicall
    return outcome.get_result()
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\package\__init__.py", line 16, in tox_package
    session.package, session.dist = get_package(session)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\package\__init__.py", line 29, in get_package
    package = acquire_package(config, session)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\package\__init__.py", line 40, in acquire_package
    path = build_package(config, session)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\package\builder\__init__.py", line 9, in build_package
    return build(config, session)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\package\builder\isolated.py", line 32, in build
    if package_venv.setupenv():
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\venv.py", line 615, in setupenv
    with self.new_action("getenv", self.envconfig.envdir) as action:
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\venv.py", line 135, in new_action
    self.envconfig.envpython,
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\config\__init__.py", line 1032, in envpython
    return self.get_envpython()
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\config\__init__.py", line 1040, in get_envpython
    return self.envbindir.join(name)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\config\__init__.py", line 1027, in envbindir
    return self.get_envbindir()
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\config\__init__.py", line 1014, in get_envbindir
    isinstance(self.python_info, NoInterpreterInfo)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\config\__init__.py", line 1053, in python_info
    return self.config.interpreters.get_info(envconfig=self)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\interpreters\__init__.py", line 33, in get_info
    executable = self.get_executable(envconfig)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\interpreters\__init__.py", line 27, in get_executable
    exe = self.hook.tox_get_python_executable(envconfig=envconfig)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\callers.py", line 208, in _multicall
    return outcome.get_result()
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\pluggy\callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\interpreters\windows\__init__.py", line 14, in tox_get_python_executable
    spec, path = base_discover(envconfig)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\interpreters\common.py", line 17, in base_discover
    cur_spec = exe_spec(discover, envconfig.basepython)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\interpreters\via_path.py", line 38, in exe_spec
    info = get_python_info(python_exe)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\site-packages\tox\interpreters\via_path.py", line 66, in get_python_info
    universal_newlines=True,
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\lib-python\3\subprocess.py", line 744, in __init__
    restore_signals, start_new_session)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\lib-python\3\subprocess.py", line 1045, in _execute_child
    startupinfo)
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\lib_pypy\_winapi.py", line 321, in CreateProcess
    raise _WinError()
  File "C:\hostedtoolcache\windows\PyPy\3.6.9\x86\lib_pypy\_winapi.py", line 30, in _WinError
    raise excep
OSError: [WinError 87] The parameter is incorrect
Error: Process completed with exit code 1.

Not sure if this is due to my use of python -m tox and whether this has changed because of a more recent version of tox or because of a recent change in PyPy.

[jayvdb]

It would be helpful if we can get a verbose log using python -m tox -vvvv.

It would be interesting what D:\ drive is being mapped to. tox has a few open issues about UNC paths, and its internal logic does some conversion from paths to UNC paths which can trigger these bugs.

[gaborbernat]

Unfortunately I don't have a Windows machine easily available,

Takes a while to download, but you can get one, for free, via https://developer.microsoft.com/en-us/windows/downloads/virtual-machines/. Pick your favorite virtualization machine type, and away you go.

[pganssle]

It would be helpful if we can get a verbose log using python -m tox -vvvv.

This is actually failing before tox does anything, AFAICT? tox -vvv log doesn't show any difference.

It looks like tox is failing when trying to detect the version of Python in use.

[pganssle]

Further investigation:

1. Pinning to tox 3.16 and 3.17 don't help, which makes me think that the issue is related to PyPy.
2. Looking back at my old logs, this used to work with PyPy 3.6.9 with version number 7.3.1. The current version is 7.3.2.

[gaborbernat]

Can you set VIRTUALENV_VERBOSE=5 and add it to passenv? And run it then?

[pganssle]

@gaborbernat From @mattip on #pypy:

what should os.path.exists('') produce: True or False?
it turns out that is the problem with pypy7.3.2 on windows
the correct answer is False, but somehow we changed it to True

So this appears to be a bug in PyPy. Not sure if it's possible or worth working around in tox.

[mattip]

Tracked this down to a regression in os.path.exists('') which breaks the discoverers in tox/interpreters/common.py

[mattip]

Is there a way to set TOX_DISCOVER to something lke pypy3-7.3.2-messed-this-up-and-now-I-need-to-work-around-it to avoid the default value? Or will that break other things?

[gaborbernat]

I'd expect to mess things up, but one can try 🤔

[pganssle]

Created an issue for this upstream: https://foss.heptapod.net/pypy/pypy/-/issues/3331

Would be interested to know if someone comes up with a better workaround than "skip PyPy3 on Windows", though.

[hugovk]

One workaround is to temporarily test PyPy without tox: https://github.com/jmoiron/humanize/pull/174/files

[dmerejkowsky]

Upstream bug is fixed. We just have to wait for the next pypy release :)

[AleksaC]

Running something along the lines of tox --discover $(which python) seems to work. I don't know if it could backfire in some way but it works fine in Azure Pipelines with bash step. Works in Github Actions as well if you change the shell to bash.

[webknjaz]

FTR this is reproducible on GHA under a Windows PyPy3 job with TOXENV=python set: https://github.com/cherrypy/cheroot/runs/1404214083?check_suite_focus=true#step:6:18.
It's tox --notest. The most interesting part is that tox printed out a traceback but completed with a successful return code.

[hugovk]

PyPy 7.3.3 has now been released with a fix.

• https://morepypy.blogspot.com/2020/11/pypy-733-triple-release-python-37-36.html

https://github.com/actions/setup-python/issues/163 is a request to add it to GHA.

[hugovk]

GHA now has PyPy 7.3.3, so tox+PyPy+Windows works now: https://github.com/actions/setup-python/issues/163#issuecomment-739907048

e.g. https://github.com/jmoiron/humanize/pull/185

Thanks everyone!