Tox: WindowsError: [Error 1920] The file cannot be accessed by the system

[sajith]

I am working on setting up GitHub Actions to test zfec library. Zfec uses tox, and I have this in tox.ini:

[tox]
envlist = py{27,36,37,38,py2,py3}
skip_missing_interpreters = True

[testenv]
deps = .[test]
usedevelop = True

commands =
     python setup.py test

Running tox works on Linux and macOS as expected, but on Windows there's an error:

vagrant@DESKTOP-JCEFROP C:\Users\vagrant\zfec-sajith>python --version
Python 2.7.18

vagrant@DESKTOP-JCEFROP C:\Users\vagrant\zfec-sajith>tox
py27 create: C:\Users\vagrant\zfec-sajith\.tox\py27
py27 installdeps: .[test]
py27 develop-inst: C:\Users\vagrant\zfec-sajith
py27 installed: DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop suppor
t for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 wil
l remove support for this functionality.,attrs==20.3.0,Automat==20.2.0,constantly==15.1.0,hyperlink==20.0.1,idna==2.10,incremental==17.5.0,pathlib2==2.3.5,PyHamcrest==1.10.1,pyutil=
=3.3.0,scandir==1.10.0,setuptools-trial==0.6.0,six==1.15.0,Twisted==20.3.0,typing==3.7.4.3,-e git+https://github.com/sajith/zfec@b2e9d11299b780765fa7347bfb40804e1d033968#egg=zfec,zo
pe.interface==5.2.0
py27 run-test-pre: PYTHONHASHSEED='65'
py27 run-test: commands[0] | python setup.py test
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to us
e tox.
running egg_info
writing dependency_links to zfec.egg-info\dependency_links.txt
writing requirements to zfec.egg-info\requires.txt
writing zfec.egg-info\PKG-INFO
writing entry points to zfec.egg-info\entry_points.txt
writing top-level names to zfec.egg-info\top_level.txt
reading manifest file 'zfec.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
writing manifest file 'zfec.egg-info\SOURCES.txt'
running build_ext
copying build\lib.win-amd64-2.7\zfec\_fec.pyd -> zfec
test_basic (zfec.test.test_zfec.Cmdline) ... ok
test_stdin (zfec.test.test_zfec.Cmdline) ... ok
test_bad_args_dec (zfec.test.test_zfec.EasyFecTest) ... ok
test_random (zfec.test.test_zfec.EasyFecTest) ... ok
test_small (zfec.test.test_zfec.EasyFecTest) ... ok
test_filefec_all_shares (zfec.test.test_zfec.FileFec) ... ok
test_filefec_all_shares_1_b (zfec.test.test_zfec.FileFec) ... ok
test_filefec_all_shares_2 (zfec.test.test_zfec.FileFec) ... ok
test_filefec_all_shares_2_b (zfec.test.test_zfec.FileFec) ... ok
test_filefec_all_shares_3 (zfec.test.test_zfec.FileFec) ... ok
test_filefec_all_shares_3_b (zfec.test.test_zfec.FileFec) ... ok

[...]

----------------------------------------------------------------------
Ran 28 tests in 1.484s

OK
py27 finish: run-test  after 3.84 seconds
py27 start: run-test-post
py27 finish: run-test-post  after 0.00 seconds
python (C:\Python27\python.EXE) is {u'sysplatform': u'win32', u'executable': u'C:\\Python27\\python.EXE', u'version': u'2.7.18 (v2.7.18:8d21aa21f2, Apr 20 2020, 13:25:05) [MSC v.150
0 64 bit (AMD64)]', u'version_info': [2, 7, 18, u'final', 0], u'implementation': u'CPython', u'is_64': True, u'extra_version_info': None}
py36 uses None
py36 start: getenv C:\Users\vagrant\zfec-sajith\.tox\py36
py36 cannot reuse: -r flag
py36 create: C:\Users\vagrant\zfec-sajith\.tox\py36
SKIPPED: InterpreterNotFound: python3.6
py36 finish: getenv C:\Users\vagrant\zfec-sajith\.tox\py36 after 0.01 seconds
___________________________________ summary ___________________________________
  py27: commands succeeded
SKIPPED:  py36: InterpreterNotFound: python3.6
ERROR:   py37: undefined
ERROR:   py38: undefined
ERROR:   pypy2: undefined
ERROR:   pypy3: undefined
Traceback (most recent call last):
  File "c:\python27\lib\runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "c:\python27\lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "C:\Python27\Scripts\tox.exe\__main__.py", line 7, in <module>
  File "c:\python27\lib\site-packages\tox\session\__init__.py", line 44, in cmdline
    main(args)
  File "c:\python27\lib\site-packages\tox\session\__init__.py", line 69, in main
    exit_code = session.runcommand()
  File "c:\python27\lib\site-packages\tox\session\__init__.py", line 197, in runcommand
    return self.subcommand_test()
  File "c:\python27\lib\site-packages\tox\session\__init__.py", line 225, in subcommand_test
    run_sequential(self.config, self.venv_dict)
  File "c:\python27\lib\site-packages\tox\session\commands\run\sequential.py", line 9, in run_sequential
    if venv.setupenv():
  File "c:\python27\lib\site-packages\tox\venv.py", line 615, in setupenv
    with self.new_action("getenv", self.envconfig.envdir) as action:
  File "c:\python27\lib\site-packages\tox\venv.py", line 135, in new_action
    self.envconfig.envpython,
  File "c:\python27\lib\site-packages\tox\config\__init__.py", line 1032, in envpython
    return self.get_envpython()
  File "c:\python27\lib\site-packages\tox\config\__init__.py", line 1040, in get_envpython
    return self.envbindir.join(name)
  File "c:\python27\lib\site-packages\tox\config\__init__.py", line 1027, in envbindir
    return self.get_envbindir()
  File "c:\python27\lib\site-packages\tox\config\__init__.py", line 1014, in get_envbindir
    isinstance(self.python_info, NoInterpreterInfo)
  File "c:\python27\lib\site-packages\tox\config\__init__.py", line 1053, in python_info
    return self.config.interpreters.get_info(envconfig=self)
  File "c:\python27\lib\site-packages\tox\interpreters\__init__.py", line 33, in get_info
    executable = self.get_executable(envconfig)
  File "c:\python27\lib\site-packages\tox\interpreters\__init__.py", line 27, in get_executable
    exe = self.hook.tox_get_python_executable(envconfig=envconfig)
  File "c:\python27\lib\site-packages\pluggy\hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "c:\python27\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\python27\lib\site-packages\pluggy\manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "c:\python27\lib\site-packages\pluggy\callers.py", line 208, in _multicall
    return outcome.get_result()
  File "c:\python27\lib\site-packages\pluggy\callers.py", line 81, in get_result
    _reraise(*ex)  # noqa
  File "c:\python27\lib\site-packages\pluggy\callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "c:\python27\lib\site-packages\tox\interpreters\windows\__init__.py", line 33, in tox_get_python_executable
    return check_with_path(candidates, spec)
  File "c:\python27\lib\site-packages\tox\interpreters\via_path.py", line 24, in check_with_path
    cur_spec = exe_spec(path, base)
  File "c:\python27\lib\site-packages\tox\interpreters\via_path.py", line 38, in exe_spec
    info = get_python_info(python_exe)
  File "c:\python27\lib\site-packages\tox\interpreters\via_path.py", line 66, in get_python_info
    universal_newlines=True,
  File "c:\python27\lib\subprocess.py", line 394, in __init__
    errread, errwrite)
  File "c:\python27\lib\subprocess.py", line 644, in _execute_child
    startupinfo)
WindowsError: [Error 1920] The file cannot be accessed by the system

Output of tox -rvv: tox-output.txt

[sajith]

In case it helps, I am using this Vagrant box for local testing on Windows. I do not have a Windows machine.

[gaborbernat]

For what's worth setuptools deprecated this way of invocation (python setup.py test) a while back so you'd probably want to replace that with something else as the output posted by you says:

WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to us
e tox.

[sajith]

@gaborbernat Good idea, thank you!

[sajith]

Switching from python setup.py test to trial zfec/test/test_zfec.py did not make a difference on Windows. The issue reported above still stands.

[gaborbernat]

Please post the output of tox -rvv.

[sajith]

The file attached with the bug report above, will that do?

It's way down in the bottom and easy to miss. Sorry about that. :-)

[gaborbernat]

The issue is this one:

SKIPPED:  py36: InterpreterNotFound: python3.6

While we're trying to discover a such interpreter on your machine something goes bad and the executable we're trying to run throws WindowsError: [Error 1920] The file cannot be accessed by the system. To find out why you'd need to poke in here:

File "c:\python27\lib\site-packages\tox\interpreters\via_path.py", line 38, in exe_spec
    info = get_python_info(python_exe)
  File "c:\python27\lib\site-packages\tox\interpreters\via_path.py", line 66, in get_python_info

Add some print messages there to see the path, and then looking on that file understand why it's failing with such.

[sajith]

Do you mean python_exe? Seems that it is the "system" Python (C:\Python27\python.EXE) which runs tox.

And get_python_info() returned {u'sysplatform': u'win32', u'executable': u'C:\\Python27\\python.EXE', u'version': u'2.7.18 (v2.7.18:8d21aa21f2, Apr 20 2020, 13:25:05) [MSC v.1500 64 bit (AMD64)]', u'version_info': [2, 7, 18, u'final', 0], u'implementation': u'CPython', u'is_64': True, u'extra_version_info': None}, which does not look suspicious.

I removed py36 from tox.ini, and tox is still failing in the same manner. I installed Python3.9 using chocolatey and re-installed tox, and I still had a very similar error:

[...]
  py27: commands succeeded
SKIPPED:  py36: InterpreterNotFound: python3.6
ERROR:   py37: undefined
ERROR:   py38: undefined
ERROR:   pypy2: undefined
ERROR:   pypy3: undefined
Traceback (most recent call last):
  File "c:\python39\lib\site-packages\tox\interpreters\__init__.py", line 25, in get_executable
    return self.name2executable[envconfig.envname]
KeyError: 'py37'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Python39\Scripts\tox.exe\__main__.py", line 7, in <module>
  File "c:\python39\lib\site-packages\tox\session\__init__.py", line 44, in cmdline
    main(args)
  File "c:\python39\lib\site-packages\tox\session\__init__.py", line 69, in main
    exit_code = session.runcommand()
  File "c:\python39\lib\site-packages\tox\session\__init__.py", line 197, in runcommand
    return self.subcommand_test()
  File "c:\python39\lib\site-packages\tox\session\__init__.py", line 225, in subcommand_test
    run_sequential(self.config, self.venv_dict)
  File "c:\python39\lib\site-packages\tox\session\commands\run\sequential.py", line 9, in run_sequential
    if venv.setupenv():
  File "c:\python39\lib\site-packages\tox\venv.py", line 615, in setupenv
    with self.new_action("getenv", self.envconfig.envdir) as action:
  File "c:\python39\lib\site-packages\tox\venv.py", line 135, in new_action
    self.envconfig.envpython,
  File "c:\python39\lib\site-packages\tox\config\__init__.py", line 1032, in envpython
    return self.get_envpython()
  File "c:\python39\lib\site-packages\tox\config\__init__.py", line 1040, in get_envpython
    return self.envbindir.join(name)
  File "c:\python39\lib\site-packages\tox\config\__init__.py", line 1027, in envbindir
    return self.get_envbindir()
  File "c:\python39\lib\site-packages\tox\config\__init__.py", line 1014, in get_envbindir
    isinstance(self.python_info, NoInterpreterInfo)
  File "c:\python39\lib\site-packages\tox\config\__init__.py", line 1053, in python_info
    return self.config.interpreters.get_info(envconfig=self)
  File "c:\python39\lib\site-packages\tox\interpreters\__init__.py", line 33, in get_info
    executable = self.get_executable(envconfig)
  File "c:\python39\lib\site-packages\tox\interpreters\__init__.py", line 27, in get_executable
    exe = self.hook.tox_get_python_executable(envconfig=envconfig)
  File "c:\python39\lib\site-packages\pluggy\hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "c:\python39\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\python39\lib\site-packages\pluggy\manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "c:\python39\lib\site-packages\pluggy\callers.py", line 208, in _multicall
    return outcome.get_result()
  File "c:\python39\lib\site-packages\pluggy\callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "c:\python39\lib\site-packages\pluggy\callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "c:\python39\lib\site-packages\tox\interpreters\windows\__init__.py", line 33, in tox_get_python_executable
    return check_with_path(candidates, spec)
  File "c:\python39\lib\site-packages\tox\interpreters\via_path.py", line 24, in check_with_path
    cur_spec = exe_spec(path, base)
  File "c:\python39\lib\site-packages\tox\interpreters\via_path.py", line 38, in exe_spec
    info = get_python_info(python_exe)
  File "c:\python39\lib\site-packages\tox\interpreters\via_path.py", line 62, in get_python_info
    proc = subprocess.Popen(
  File "c:\python39\lib\subprocess.py", line 947, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "c:\python39\lib\subprocess.py", line 1416, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
OSError: [WinError 1920] The file cannot be accessed by the system

Looks like it is _winapi.CreateProcess() that is failing, and it gets arguments executable: None, args: C:\Users\vagrant\AppData\Local\Microsoft\WindowsApps\python3.7.EXE C:\Python39\Lib\site-packages\tox\helper\get_version.py, env: None, cwd: None.

Now, I don't know where C:\Users\vagrant\AppData\Local\Microsoft\WindowsApps\python3.7.EXE is coming from -- I have only installed Python 2.7 and 3.9 in this box. But some additional Pythons also exist in this box, and they cannot be executed:

vagrant@DESKTOP-JCEFROP C:\Users\vagrant\AppData\Local\Microsoft\WindowsApps> dir
 Volume in drive C has no label.
 Volume Serial Number is 1483-A540

 Directory of C:\Users\vagrant\AppData\Local\Microsoft\WindowsApps

11/07/2020  08:53 PM    <DIR>          Microsoft.DesktopAppInstaller_8wekyb3d8bbwe
11/07/2020  08:53 PM    <DIR>          Microsoft.MicrosoftEdge_8wekyb3d8bbwe
11/07/2020  08:53 PM                 0 MicrosoftEdge.exe
11/07/2020  08:53 PM                 0 python.exe
11/07/2020  08:53 PM                 0 python3.7.exe
11/07/2020  08:53 PM                 0 python3.exe
               4 File(s)              0 bytes
               2 Dir(s)  113,001,963,520 bytes free

vagrant@DESKTOP-JCEFROP C:\Users\vagrant\AppData\Local\Microsoft\WindowsApps>.\python3.7.exe
The system cannot execute the specified program.

Removing py37 from tox's list seems to solve my original problem. But I guess there's another issue here: tox could perhaps check that the Python it is attempting to execute is indeed executable?

[exarkun]

I don't think this will help with this particular problem but:

Switching from python setup.py test to trial zfec/test/test_zfec.py did not make a difference on Windows. The issue reported above still stands.

Using trial with paths is problematic. The preferred way to run a test module like this would be trial zfec.test.test_zfec.

[gaborbernat]

Now, I don't know where C:\Users\vagrant\AppData\Local\Microsoft\WindowsApps\python3.7.EXE is coming from

Someone installed it from Microsoft store, see https://devblogs.microsoft.com/python/python-in-the-windows-10-may-2019-update. Sadly store 3.7 is something we (or virtualenv - but version 3.8 should work) don't support.

But I guess there's another issue here: tox could perhaps check that the Python it is attempting to execute is indeed executable?

We should. However, I think the issue is not that it's not executable, but rather we don't have the right to execute it, or some variation of that..

[gaborbernat]

Closing per my comment above nothing more we can do.