Vscode-python: Detect virutal envs recursively in all sub-directories in the workspace

[PooyaJln]

This issue is similar to #2470 but the solution doesn't apply to me.
I tried changing the "python.venvPath": "${workspaceFolder}/venv" in setting.json but it didn't help.
As it can be seen in the screenshot, I can use the virtual environment using terminal.

Environment data

• VS Code version: 1.45.1
• Extension version (available under the Extensions sidebar): 2020.5.80290
• OS and version: ubuntu 18.04 (Linux x64 4.15.0-101-generic)
• Python version (& distribution if applicable, e.g. Anaconda): python 3.6.9
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): venv
• Relevant/affected Python packages and their versions: XXX
• Relevant/affected Python-related VS Code extensions and their versions: XXX
• Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info #3977): true
• Value of the python.languageServer setting: "Microsoft"

Expected behaviour

Clicking on the interpretor on the left bottom should show all available interpretors including virtual environments.

Actual behaviour

virtual environment are not detected.

Steps to reproduce:

1. click on the interpretor selection on the left bottom
   

Logs

[karthiknadig]

From the screen shot it looks like the venv is under ${workspaceFolder}/test_proj/venv. Can you confirm that the location is set correctly?

[PooyaJln]

Tha't right.
If I open the project folder itself (here the 'test-project') VScode detects the virtual environment but if I open parent folder of 'test_project' that contains several project folders it can't detect he environment.

[mattburgess]

Jumping on this one...I'm happy to raise a separate issue, but it looks related to me.

In my case, I've got a pipenv virtualenv configured in the default location (~/.local/share/virtualenvs). Adding the root folder of the project to a new workspace results in it picking the default system Python interpreter, /usr/bin/python.

As it's in a default location, I'd expect the extension to automatically detect my root folder belongs to a pipenv virtualenv and automatically select the relevant interpreter for me.

As that didn't work, I added "python.venvPath": "~/.local/share/virtualenvs" to each of my user, workspace and project's settings.json in turn, but none had the desired effect. The only way I seem to be able to get the right interpreter set is to select it manually.

The output window shows the following on initial startup:

> conda --version
> pyenv root
> python3.7 ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python3.6 ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python3 ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python2 ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> conda info --json
> python ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> python ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import sys;print(sys.executable)"
> /usr/bin/python ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import jupyter"
> /usr/bin/python ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import jupyter"
> /usr/bin/python ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import notebook"
> /usr/bin/python ~/.vscode-oss/extensions/ms-python.python-2020.5.86398/pythonFiles/pyvsc-run-isolated.py -c "import notebook"
Starting Jedi Python language engine.
Python interpreter path: /usr/bin/python

At no point does it even appear to be trying to see if the project might be a pipenv style project. I'd have thought it might want to run things like pipenv --version to see if pipenv is available, and then pipenv --venv to see if the current directory is in a pipenv project, and finally pipenv --py to find the interpreter to use.

This is all on vscode-1.45.1 and extension version 2020.5.86398

[karthiknadig]

@PooyaJln I don;t understand the issue, in your comment you mentioned that "python.venvPath": "${workspaceFolder}/venv" which is incorrect since the venv is actually under ${workspaceFolder}/test_proj/venv. Are there different venv's under each folder?

@mattburgess We don't search for pipenv on startup. This is intentional, since doing environment discovery with pipenv can have unintended side effects. So until you hit select interpreter we don't look for pipenvs.

[mattburgess]

@karthiknadig that's interesting. Reading https://code.visualstudio.com/docs/python/environments#_where-the-extension-looks-for-environments I was under the impression that pipenv interpreters would be located automatically.

A pipenv environment for the workspace folder. If one is found, then no other interpreters are searched for or listed as pipenv expects to manage all aspects.

That, coupled with:

You can also manually specify an interpreter if Visual Studio Code does not locate it automatically.

suggested to me that I shouldn't have to manually select my interpreter. I wonder if the docs could be clarified a bit to specifically point out that pipenv environments will not be located automatically. I'd be interested to know what the "unintended side effects" are of doing environment discovery with pipenv although that probably could be omitted from the docs :)

[PooyaJln]

What I discovered later was that VScode finds the virtual environment if I open the project's folder itself but if I open a folder that contains several project folders it can't detect it.
What I understood from the setting environment guide was that it should be possible by using the correct ${} variable.
I also tried
"python.venvPath": "${fileDirname}/venv/bin/python3"
Which didn't work either.

If I'm wrong and the default behaviour is to only detect the virtual environment inside the project folder itself, then it's not an issue and we can close this thread.

[yuvadm]

I absolutely agree with @mattburgess comments, my understanding of the documentation is the same.

Furthermore, when using pipenv, there's a rough edge in how VS code saves the venv settings.

Assume two (or more) developers are working on a shared project, each using pipenv. Once each developer checks out the project, and selects the right interpreter, each will have a different value in settings.json.

One will have:

{
    "python.pythonPath": "/home/userAAA/.local/share/virtualenvs/myproj-AaBbCcDd/bin/python"
}

While the other will have:

{
    "python.pythonPath": "/home/userBBB/.local/share/virtualenvs/myproj-ZzYyXxWw/bin/python"
}

This means they cannot commit a shared workspace settings.json that will select each pipenv automatically according to the myproj name. I would expect:

{
    "python.venvPath": "~/.local/share/virtualenvs/",
}

to handle this. But otherwise, the current behavior means that both developers will always have a mismatch - and thus a diff - in their settings and will have to set their VCS to ignore that file.

[luabud]

Thanks for bringing this up, folks! I opened https://github.com/microsoft/vscode-docs/issues/3789 so we can update our docs to reflect that 😊

@yuvadm our plan is to deprecate the pythonPath setting, so you'll still be able to select interpreters per workspace but that info won't be stored in the settings.json file. We have this change on as an experiment, you can learn more about it here: https://github.com/microsoft/vscode-python/wiki/AB-Experiments#experiment---deprecatepythonpath

[karthiknadig]

@PooyaJln With python.venvPath we only look in the immediate sub-directories. So if the venv folder is two levels down we will not automatically detect it. This would be an enhancement to how we detect virtual envs in the workspace.

[luabud]

We decided we won't look for environments in sub-directories for performance reasons. The problem that was mentioned in this comment will be addressed once we ship our python.pythonPath changes to all users (#11015).