Vscode-python: `pyvsc-run-isolated.py` removes pip "editable" installations from `sys.path`, breaks pytest

[akukuq]

In #14014, pythonFiles/pyvsc-run-isolated.py script has been modified to remove all occurrences of cwd or '' from sys.path. Previously, this script just blindly replaced (and before that it removed) sys.path[0]. The purpose of #14014 was to handle situations in which cwd is not the first item in sys.path.

The unintended side effect of #14014 arises when a package residing in cwd is installed with pip install -e . (as an editable install). Such installation causes cwd to be added at the end of sys.path. In these situations, cwd exists twice in sys.path: first as '' and second as cwd. Removing both occurrences breaks everything that needs to import the package installed as editable.

In my case what is broken is running pytest tests from the GUI. Side note: why is pytest run using the isolate script?

[karthiknadig]

We have marked this issue as "needs decision" to make sure we have a conversation about your issue. We plan to leave this request open for at least a month to see how many 👍 votes the opening comment gets to help us make our decision.

[EvilMav]

Same her: running tests from GUI is broken on a package installed using 'setup.py develop'. Likely related, pylint is also broken, marking same package imports as not found.

[ryannk]

Sorry, I am still fairly new to python. What is the expected way for the target package to be installed locally when running pytest? From the pytest documentation it seems like installing pip install -e is suggested.

[emj1991]

Not sure if this is related, but the latest extension (v2020.10.33229234) also breaks my tests. When running via GUI I get "Cannot read property 'testsuites' of null". Running pytest from command line works fine. Downgrading to previous extension restores normal operation of my tests.

[karthiknadig]

@emj1991 That is an unrelated issue. Please file a separate bug.

[mrdavidlaing]

For me; pytest and pylint were finding finding modules using the PYTHONPATH=. setting in my ${workspaceFolder}/.env file. (as suggested here)

The behaviour introduced in #14014 that removes $(cwd) from sys.path breaks this setting; causing both to fail.

[akukuq]

@karthiknadig , I am glad to see that this has been approved for fixing in next release.

I also wanted to start a discussion on how this should be fixed. I see two ways:

• change behavior of the isolate script to remove at most one instance of '' or cwd from sys.path
• run tests and linters without the use of the isolate script (The question being: what is the purpose of running those via the isolate script? Is running pytest or pylint directly in the workspace root considered a risk?)

[brettcannon]

@akukuq we're still trying to think through how best to handle this because we don't want to drop the functionality (it helps a lot of users who accidentally shadow stdlib modules which is the primary motivator for this). And it's made more complicated by the fact that the location of the path entry could be anywhere as the Windows embedded install does not put it in sys.path[0] as is tradition.

So maybe your option 1, but we have not had a chance to discuss and think through the ramifications yet.

[falkben]

I've had to disable pylint linting in my projects, which is annoying, since we have it turned on in our CI/CD, so disabling it in my editor is definitely not ideal.

Another option maybe -- ability to choose whether to run in isolated mode?

Edit: also interesting that "Debug Test" adornment works but not "Run Test"

[akukuq]

@brettcannon , if I understand you correctly the motivation of the isolate script is to make tests/linters function even if they would not work when run from the terminal emulator, e.g. due to name clashes between files in workspace root and stdlib modules. If this is so, I would like to offer my view on this matter for your consideration:

Before coming to VSCode I have been a PyCharm user (many of my colleagues still are). For me, the most important advantage of using VSCode over PyCharm is that VSCode puts the user in charge of far more aspects of workspace configuration than PyCharm does. Moreover, VSCode is configured through text files and allows the user to configure tests/linters/whatnot using the standard configuration files of the respective programs. These files can than be committed to repository, reused by other users (even using other IDEs) and (very importantly) in CI context. This has huge value when it comes to educating users how to use all these tools without VSCode. VSCode encouraged me to learn things and made me a better programmer, PyCharm prevents users from learning.

To illustrate how important this can be I can say the following: It is very common for my colleagues who exclusively use PyCharm to have problems with correct configuration of their projects, to the point that some of the software they write works only when run from within PyCharm! I recall that one piece of software had installation instructions that started with "1. Install PyCharm".

For this reason I feel that IDEs should not try to fix what users have broken. This makes them ignorant of how projects should be organized. In my opinion testing and linting should be run directly in user's environment.

[mtaron]

In addition to pytest, this regression seems to effect notebooks as well. I get module not found errors when importing a project in the same folder (i.e. using cwd) in a notebook. Import works fine when downgrading to 2020.9.114305.

[smcoll]

i have another example of a failing case: running tests for PEP-420 native namespace packages. Here's a repo to demonstrate: https://github.com/smcoll/vscode-python-14570