Notebook: Notebook cannot load the environment variables automatically

[sw-gong]

I set my envrionment varibale in ~/.bashrc: export QT_API=pyqt

It shows correctly in the terminal, when I ran export.

However, if I launch Jupyter Notebook, this environment variable is not set by using ! export in side Notebook.

[takluyver]

! runs system commands in a subprocess. Subprocesses can't affect environment variables in their parent process. You can modify environment variables in Python using os.environ.

[sw-gong]

@takluyver

The goal of running this command ! was not for adding new environment variables but showing the current environment variables, or to show the issues I met. I know how to add environment variables in Jupyter Notebook. What I am curious is why Jupyter Notebook cannot load environment variables I have already set automatically.

Let's say it again for simplicity.

Jupyter Notebook cannot launch environment variables automatically after I set environment variables in ~/.bashrc. But those variables can take effect in python.

[takluyver]

Oh, sorry, I misread your issue.

Are you launching Jupyter Notebook from a terminal, or from a GUI launcher of some kind?

.bashrc affects bash, and then environment variables are inherited by anything you run from bash. So it won't affect something launched from a GUI shortcut, because that hasn't gone through bash. On a Linux system, you can set environment variables in .profile, which is run at login, so it should affect everything.

[sw-gong]

Hi @takluyver

Yes, I launch Jupyter Notebook from a terminal. Indeed, I assume the environment variables set in .bashrc should be inherited when I launch Notebook in the terminal. But I don't know why it doesn't.

[sw-gong]

Plus, I'm using zsh (to be specific, oh-my-zsh). Although I added environment variables inside ~/.bashrc, I added source ~/.bashrc in ~/.zshrc. I suppose those environment variables should take effect as the same when I launch Jupyter Notebook in the terminal, right?

[otmezger]

For me it works:

if I add this to my ~/.bash_profile export JUPYTER_PATH="/Users/me/whatever/:$JUPYTER_PATH", then the specified path is in the python path from within jupyter. I can confirm by running sys.path()

I can also set my own environment variables in ~/.bash_profile, start a new console (iTerm2 in mac os), enter the enviroment in conda, and lauching jupyter manually from there.

[dirac-dc]

I have the exact same issue, where did it read the initial variables in its path if it doesn't load from .bash_profile?

[kevin-bates]

Are your environments completely empty or just missing some expected items? If only missing expected items, those items need to be _sourced_ into your runtime environment prior to launching the Notebook server. I suspect that's not the issue per the previous responses.

If the environment is (nearly) empty with just a couple items present, then you might be using either of the Gateway projects (Kernel Gateway, Enterprise Gateway). These projects purposely reset the env because the launching server's env is not that of the notebook's user.

Another possibility is that your NB application uses its own kernel manager that is controlling its env similar to the gateways, since the only way the server's env is NOT passed along to the kernel process is if env is present in the keyword arguments. Here's the applicable code.

[guyskk]

@sw-gong do you start jupyter notebook by systemd or launchd? If yes then it's non-login shell which is different with your terminal.

I meet the same problem when start jupyter by launchd, and fix it by modify the startup script:

#!/bin/bash

set -ex
source ~/.bashrc  # ADD THIS LINE

$HOME/anaconda3/bin/jupyter notebook --no-browser $HOME/myjupyter

[edenbaus]

@sw-gong do you start jupyter notebook by systemd or launchd? If yes then it's non-login shell which is different with your terminal.

I meet the same problem when start jupyter by launchd, and fix it by modify the startup script:

#!/bin/bash

set -ex
source ~/.bashrc  # ADD THIS LINE

$HOME/anaconda3/bin/jupyter notebook --no-browser $HOME/myjupyter

@guyskk Thank you!

I just setup a new compute engine image with jupyter labs as a service with systemd. I kept scratching my head why my python notebooks weren't picking up the right environmental variables I setup in the user's ~/.bashrc file.

[PedramBashiri]

I faced the same problem (using os.environ['VARIABLE_NAME'], changed an environment variable, but the value would'n change in the jupyter notebook). Restarting the kernel or the whole Jupyter didn't help. Had to restart anaconda (from which I had started Jupyter) to get the correct value of the variable.

[syahdeini]

I also have the problem,
I change my .env file and it can not replace the old one,
it is somehow cache my environment variable. Even though I try to reload it using dotenv, restart jupyter notebook or anything.

[yujund]

still no remedy to this problem?

[kevin-bates]

Hi @yujund, I don't think there's anything to resolve here, at least that's obvious. I personally cannot reproduce the issue and suspect there's a disconnect somewhere. Perhaps you could share your expectation?

Here's what I just did...
From a MacOS terminal window, I set an environment variable named ISSUE to 3704 and launch notebook

export ISSUE=3704
jupyter notebook

I then start a python kernel and access the environment variable from the cell...

Same goes for .bashrc. Unless it's sourced _prior_ to launching the Notebook server, any environment variables exported in .bashrc are not available to a launched kernel. Sure, if I create a sub-shell using bash, then .bashrc is sourced by that action and you _should_ see those variables from the kernel (I do), but the act for running jupyter notebook or launching a kernel, does _source_ a new shell. As a result, the behavior should be the following - which I believe is consistent across all responses:

_Whatever environment you have when launching the Notebook server, is the environment you should see from within a launched kernel._

There are some variations to this and you'll see differences with the following:

1. You're using a Gateway server - which only propagates the server's env via configuration.
2. You've added an env stanza to your kernel's kernel.json file, in which case you'll see additional environment variables in your kernel's environment.
3. You're actually launching a shell script from which you can do anything to set up your kernel's env prior to invoking the kernel from that script.

[yujund]

@kevin-bates Thanks for your information. It solves the problem.

[kevin-bates]

Thanks for the update. I'm going to close this as-designed. If others have issues we can talk about re-opening at that time.