Virtualenv: [20.0.10] windows store support

[asottile]

1362 claims support however it does not appear to work as desired:

(venv) C:\Users\IEUser\astpretty>virtualenv v -vvv --with-traceback
390 setup logging to NOTSET [DEBUG report:43]
468 could not create app data folder C:\Users\IEUser\AppData\Local\pypa\virtualenv due to FileNotFoundError(2, 'The system cannot find the path specified') [INFO app_data:54]
577 created temporary app data folder C:\Users\IEUser\AppData\Local\Temp\tmpaqvhomd3 [DEBUG app_data:32]
624 find interpreter for spec PythonSpec(path=c:\users\ieuser\astpretty\venv\scripts\python.exe) [INFO builtin:44]
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1776.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1776.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\IEUser\astpretty\venv\Scripts\virtualenv.exe\__main__.py", line 9, in <module>
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\__main__.py", line 50, in run_with_catch
    run(args, options)
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\__main__.py", line 20, in run
    session = cli_run(args, options)
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\run\__init__.py", line 27, in cli_run
    session = session_via_cli(args, options)
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\run\__init__.py", line 35, in session_via_cli
    parser = build_parser(args, options)
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\run\__init__.py", line 71, in build_parser
    parser._interpreter = interpreter = discover.interpreter
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\discovery\discover.py", line 44, in interpreter
    self._interpreter = self.run()
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\discovery\builtin.py", line 33, in run
    return get_interpreter(self.python_spec, self.app_data.folder)
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\discovery\builtin.py", line 46, in get_interpreter
    for interpreter, impl_must_match in propose_interpreters(spec, app_data):
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\discovery\builtin.py", line 60, in propose_interpreters
    yield PythonInfo.from_exe(spec.path, app_data), True
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\discovery\py_info.py", line 293, in from_exe
    proposed = proposed._resolve_to_system(app_data, proposed)
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\discovery\py_info.py", line 328, in _resolve_to_system
    target = cls.from_exe(target.system_executable, app_data)
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\discovery\py_info.py", line 290, in from_exe
    proposed = from_exe(cls, app_data, exe, raise_on_error=raise_on_error, ignore_cache=ignore_cache)
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\discovery\cached_py_info.py", line 32, in from_exe
    result = _get_from_cache(cls, py_info_cache, app_data, exe, ignore_cache=ignore_cache)
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\discovery\cached_py_info.py", line 56, in _get_from_cache
    py_info = _get_via_file_cache(cls, py_info_cache, app_data, exe_path, exe)
  File "c:\users\ieuser\astpretty\venv\lib\site-packages\virtualenv\discovery\cached_py_info.py", line 68, in _get_via_file_cache
    resolved_path_modified_timestamp = resolved_path.stat().st_mtime
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1776.0_x64__qbz5n2kfra8p0\lib\pathlib.py", line 1178, in stat
    return self._accessor.stat(self)
OSError: [WinError 1920] The file cannot be accessed by the system: 'C:\\Users\\IEUser\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\\python.exe'

I'm using the windows store python 3.7.6 on windows 10

(venv) C:\Users\IEUser\astpretty>python --version --version
Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 01:54:44) [MSC v.1916 64 bit (AMD64)]

[asottile]

note also, I couldn't run --help to show the options without hitting the error :S

[gaborbernat]

@asottile sadly it's not possible to get a hold of windows store python in CI. I'll have a look, it worked right after that PR. Seems that resolve broke the world here...

[gaborbernat]

So the culprit here seems this:

PS C:\Users\traveler\git\virtualenv> python -c 'import sys; from pathlib import Path; import os; print(sys.executable); print(os.path.exists(sys.executable)); print(Path(sys.executable).exists())'
C:\Users\traveler\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\python.exe
False
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1776.0_x64__qbz5n2kfra8p0\lib\pathlib.py", line 1356, in exists
    self.stat()
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.1776.0_x64__qbz5n2kfra8p0\lib\pathlib.py", line 1178, in stat
    return self._accessor.stat(self)
OSError: [WinError 1920] The file cannot be accessed by the system: 'C:\\Users\\traveler\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.7_qbz5n2kfra8p0\\python.exe'

So we don't have enough rights to check the existence of the path, but we can run it. pathlib raises an exception, os.path.exists just returns a silent False.

cc @zooba I expect this works as designed?

[zooba]

It's fixed in 3.8, where we rewrote stat() on Windows to handle unfollowable-and-unreadable reparse points (which had never really existed before), as well as a number of other edge cases.

os.lstat() on Python 3.7 works, so if there's a way for you to use that instead, go for it. Otherwise, feel free to declare Python 3.7 from the Store as out-of-scope/unsupported - that's why we put the "experimental" marking on it.

[gaborbernat]

So I'm tempted to drop store python 3.7 in this case. However, another issue I've run into is that store python --symlinks method creates a virtual environment that cannot be executed; and only fails after creation, not during.

[gaborbernat]

Hello, a fix for this issue has been released via virtualenv 20.0.11; see https://pypi.org/project/virtualenv/20.0.11/ (https://virtualenv.pypa.io/en/latest/changelog.html#v20-0-11-2020-03-18). Please give a try and report back if your issue has not been addressed; if not, please comment here, and we'll reopen the ticket. We want to apologize for the inconvenience this has caused you and say thanks for having patience while we resolve the unexpected bugs with this new major release.