Vscode-jupyter: How to set environment variables for Python Interactive Window (aka IPython)?

[svdHero]

Hi there,

I am using version 2019.6.24221 with Anaconda 2019.07 and Python 3.7.3 under Windows 8.1.

I am working at a software project where all python files are located in a subfolder "src". Hence, I added

"terminal.integrated.env.windows": {
        "PYTHONPATH" : "${workspaceFolder}/src;${env:PYTHONPATH}"
 }

to my local workspace settings.json.
When choosing "Run Python File inTerminal" via right-click on any of my Python files, it is exectued without any problems, because all the modules in the local directory src are found.

However, when I choose "Run Current File in Python Interactive Window", I get a ModuleNotFoundError in the Python Interactive Window, because my local source code files are not found.
When executing the IPython magic command %env, I can also see that the environment variable PYTHONPATH is not set in the Python Interactive Window environment.

How can I tell IPython/Jupyter about mylocal srcsubfolder? Can I set project specific environment variables for IPython/Jupyter somewhere?

On Stackoverflow I have only found this post:
https://stackoverflow.com/questions/53275420/how-can-i-set-enviroment-variables-for-data-science-mode-of-vscode
which did not offer any solution.

[rchiodo]

I believe the only workaround is to add the value to the sys path.

Something like so:

import sys
sys.path.insert(0, "F:\\Workspace\\src")

You'd have to run this as a cell before executing other code.

microsoft/vscode-python#6842 is a similar request.

[svdHero]

Thank you for the code snippet. Can I somehow get access to the vscode variable ${workspaceFolder} from inside the cell, so that I don't have to manually set the absolute path in sys.path.insert()?

Also, copying and pasting the code snippet repeatedly is not really convenient. Are there any plans to implement microsoft/vscode-python#6842 any time soon?

Also, in which environment does Jupyter start anyway? Does it just see the user envorinment variables or is there any environment magic happening behind the scenes?

[rchiodo]

Sorry but the ${workspaceFolder} is not available within Jupyter. It knows nothing of VS code-isms.

We generally implement features based on up votes and comments. We also accept pull requests from anybody, so if you want to give it a try, I can point you in the right direction.

Jupyter starts using the environment you have selected in the bottom left. We call that the 'Python selection'.

[greazer]

Let's honor the terminal's setting for this very thing. Also let's check the other terminal settings in case there's something else interesting to do.

[StevenLi-DS]

@svdHero This might be a workaround for the time being
https://code.visualstudio.com/docs/editor/userdefinedsnippets

[svdHero]

@StevenLi-DS Thank you. I did not know about code snippets. What an awesome feature! However, in this case it doesn't help me, because code snippets don't seem to work in the "Python Interactive Window". Correct me, if I'm wrong, but I think the PIW has it's own Jupyter-IntelliSense-thingy which does not know about code snippets from VS Code.

[StevenLi-DS]

@svdHero I think you are correct and code snippets don't really work in the interactive windows. Let's wait for https://github.com/microsoft/vscode-python/issues/6842

[rchiodo]

I believe the workaround of using python.dataScience.runStartupCommands should be sufficient for this request.

[rchiodo]

Although I guess that's kind of a pain. You have to set it special for the interactive/notebook views. It should honor the terminal settings.

[crnolan]

Is there any update to this? It seems there are bunch of other issues that have been closed regarding the .env file loading for the Python Interactive window (#9472, microsoft/vscode-python#9815, microsoft/vscode-python#7016). Some of these reference microsoft/vscode-python#10255 as the underlying cause, but that has a PR merged to solve and the issue seems to remain (as of 1.48.0 on Windows 10 - using WSL2).

The .env file solution does work (add PYTHONPATH=/path/to/src/folder) so long as the path is absolute (and VS Code is restarted after updating the .env - see microsoft/vscode-python#9816). As detailed in microsoft/vscode-python#9472, if a local path is supplied, say PYTHONPATH=./test this is resolved to a temp folder, not the workspace folder. This would be much improved if the workspace folder was the root, better again if VS code variables were resolved in the .env file (making PYTHONPATH=${workspaceFolder}/src a valid .env entry). Potentially better still would be some form of source tree configuration for each folder that would could be parsed by any of the run / debug possibilities.

Apologies if I've managed to miss some kind of update here, but I've been unable to get a simple configuration to work except by supplying platform-specific absolute paths in the .env file.

[svdHero]

I totally agree with @crnolan. Any update on this?

[rchiodo]

We are not actively pursuing this suggestion at the moment.

There's two problems.

1. We don't read terminal.integrated.env.windows on startup of a python process (VS code reads this on startup of a terminal). This would likely require changes to what we pass as an environment when launching different python executables (one of those is the kernel that runs for the interactive window).
2. .env reading is using the current working directory of the process that launches the kernel (not the working directory of the actual kernel itself). This one would be fixed by changing our daemon code (we have something that launches a python process before you even start a kernel) to use the working directory of the workspace instead of the current working directory.

[crnolan]

Hi Rich, thanks for the update. If there weren't other unintended consequences, the fix for the second sounds like it'd be useful. Having said that, it still feels like a common project folder configuration setting specifically for source path that existed 'above' both the .env and terminal.integrated.env.* settings, that could then be automatically populated in both internal and external tools, is a potentially simpler way forward given that the same paths are likely required in all runtimes.

Given this seems like a fairly common scenario (a python data science project with a set of shared modules in a source tree, and a set of tests / notebooks / exploratory work in a separate folder), do you think it's worth documenting the best current solution somewhere within the python extension documentation? I just know that it took me quite some time to figure out how to make this all work. It seems right now (unless you've got better advice) that absolute paths in the .env file is the best way to go. I could imagine to make this work across different machines / platforms, one could have a user-level default .env on each machine to contain the absolute path for each machine in a platform-appropriate manner.

[rchiodo]

Not a bad idea (to mention absolute paths are required). The documentation for .env files is here:
https://code.visualstudio.com/docs/python/environments#_environment-variable-definitions-file. You should be able to submit an issue (or a PR) against the docs here: https://github.com/Microsoft/vscode-docs