Notebook: Unable to import tensorflow in jupyter (yet able to import tensorflow in python and ipython)

[cheshirecats]

I am using OSX 10.11 with homebrew python, and tensorflow GPU with cuda and cudnn installed.

Currently tensorflow imports correctly in both python and ipython:

Python 2.7.13 (default, Dec 17 2016, 23:03:43)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.dylib locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.dylib locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.dylib locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.1.dylib locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.dylib locally
>>>

Yet I am unable to import tensorflow in jupyter (I have tried both Jupyter notebook and Jupyter console):

Jupyter console 5.0.0

Python 2.7.13 (default, Dec 17 2016, 23:03:43)
Type "copyright", "credits" or "license" for more information.

IPython 5.2.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import tensorflow
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-a649b509054f> in <module>()
----> 1 import tensorflow

/usr/local/lib/python2.7/site-packages/tensorflow/__init__.py in <module>()
     22
     23 # pylint: disable=wildcard-import
---> 24 from tensorflow.python import *
     25 # pylint: enable=wildcard-import
     26

/usr/local/lib/python2.7/site-packages/tensorflow/python/__init__.py in <module>()
     58 please exit the tensorflow source tree, and relaunch your python interpreter
     59 from there.""" % traceback.format_exc()
---> 60   raise ImportError(msg)
     61
     62 # Protocol buffers

ImportError: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 28, in <module>
    _pywrap_tensorflow = swig_import_helper()
  File "/usr/local/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
ImportError: dlopen(/usr/local/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so, 10): Library not loaded: @rpath/libcudart.8.0.dylib
  Referenced from: /usr/local/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow.so
  Reason: image not found

Error importing tensorflow.  Unless you are using bazel,
you should not try to import tensorflow from its source directory;
please exit the tensorflow source tree, and relaunch your python interpreter
from there.

I have tried adding the following in ~/.bashrc and ~/.bash_profile and /etc/profile , yet it still does not work......
export CUDA_HOME=/usr/local/cuda
export DYLD_LIBRARY_PATH="$CUDA_HOME/lib:$DYLD_LIBRARY_PATH"
export PATH="$CUDA_HOME/bin:$PATH"

[gnestor]

Error importing tensorflow. Unless you are using bazel,
you should not try to import tensorflow from its source directory;
please exit the tensorflow source tree, and relaunch your python interpreter
from there.

Did you install tensorflow from source? Are you running jupyter from the ts source directory?

[cheshirecats]

Hi. That seems to be a generic error message, as I installed tf from source and I am not running from its source directory. Now the problem must be that jupyter is unable to find "@rpath/libcudart.8.0.dylib" which is part of the cuda runtime (Strangely enough both python and ipython is able to find it).

[gnestor]

Try running the following in both the notebook and ipython:

import sys
sys.executable

Odds are they are using different python executables. If this is the case, you can update your python kernelspec. See https://github.com/jupyter/notebook/issues/397 for reference.

[cheshirecats]

I tried that, and they are both using '/usr/local/opt/python/bin/python2.7' (which I think should be the homebrew python).

[takluyver]

I'd guess it's something to do with environment variables, but I don't know which. You may be better off asking in a tensorflow forum what might cause the error that you see.

[cheshirecats]

@takluyver It's a bit strange, because the tensorflow installation runs fine in my python and ipython. So it shall have something to do with the way jupyter calls the tensorflow installation.

(On the other hand, jupyter is able to import tensorflow on my Windows PC...)

[takluyver]

Right, but Jupyter just runs your code; it doesn't do anything specific to tensorflow.

This might be a job for envzigzag. Try installing that, then run !envzig in IPython (where you can use tensorflow) and !envzag in a notebook (where you can't). It should show you any differences in environment variables.

[cheshirecats]

@takluyver Thanks. As I am running python 2.7 I couldn't run it. However, I imported os and checked os.environ, and I found I have 'DYLD_LIBRARY_PATH' in my python os.environ, but not in my Jupyter console os.environ.

So this must be the cause. But I am not sure how to solve it.

[takluyver]

Are you launching the Jupyter applications from a GUI launcher? If so, they won't pick up environment variables set in .bashrc or .bash_profile. A simple workaround is to launch them from a terminal.

On Linux, you can set environment variables in .profile, which is run when you log in. I don't know how to do the same on OSX.

[cheshirecats]

@takluyver I am running from console and I have change .bashrc and .bash_profile etc... It seems other people ran into similar problems before. I tried the method in https://github.com/jupyter/notebook/issues/1290 (by ZlodeiBaal) and it works now.

[takluyver]

Glad you've got it working. I'm a bit confused about what the problem was, but my suspicion is that there's nothing we can do in Jupyter to fix it.

[kumarNith]

@gnestor Could you please explain how to update python kernelspec.

Note : As you have mentioned in previous post I have different python executables.

[gnestor]

Ok, to find the location of your kernelspecs: jupyter kernelspec list

Then, edit your python kernelspec in your editor. I think the easiest fix is to rename your python executable from an absolute path (e.g. "/Users/grant/anaconda/bin/python") to just "python".

[ghost]

@gnestor HI, I have similar problem and can not solve it. Can you explain how to edit your python kernelspec in your editor? I am confused of it. Thanks.

[gnestor]

@rylanchiu First, run jupyter kernelspec list and you will see an output like:

Available kernels:
  babel           /Users/grant/Library/Jupyter/kernels/babel
  bash            /Users/grant/Library/Jupyter/kernels/bash
  coffeescript    /Users/grant/Library/Jupyter/kernels/coffeescript
  javascript      /Users/grant/Library/Jupyter/kernels/javascript
  jskernel        /Users/grant/Library/Jupyter/kernels/jskernel
  python2         /Users/grant/Library/Jupyter/kernels/python2
  python3         /usr/local/share/jupyter/kernels/python3

To edit the spec for python3, let's see which files are in its directory by running ls /usr/local/share/jupyter/kernels/python3:

kernel.json logo-32x32.png  logo-64x64.png

Lastly, we want to open kernel.json in our text editor, there are several ways to do this:

• If you use the Atom editor, simply run this in the terminal: atom /usr/local/share/jupyter/kernels/python3/kernel.json
• To open the file in your default handler for .json files: open /usr/local/share/jupyter/kernels/python3/kernel.json
• vim: vim /usr/local/share/jupyter/kernels/python3/kernel.json

[ghost]

@gnestor Thanks for your detailed and helpful instruction! It solved my problem perfectly!

[jclevesque]

I do have the same problem as OP, and I wonder what could be done without modifying the notebook config manually?

I have en environment variable LD_LIBRARY_PATH, which is not picked up by jupyter. python3 picks it up just fine, but when launched from jupyter it's gone. They both use the same executable (/usr/bin/python3).

I launch jupyter from the terminal, same as python3.

[takluyver]

How are you launching the notebook? If you're using some kind of GUI launcher, try launching it from the terminal with jupyter notebook.

[jclevesque]

I am launching from terminal, with the command jupyter-notebook --no-browser.

[takluyver]

Is LD_LIBRARY_PATH set in the terminal before you launch? I don't think we do anything to unset it.

[jclevesque]

OK, after closing all sessions and trying again I am unable to reproduce. I can confirm that jupyter picks up the environment variables of the calling terminal.

Must have had an atypical terminal session somewhere in there. Sorry about that.

[Giccuk]

I have tried your way @gnestor, unfortunately, it did not work. So I kept searching and found an answer from StackOverflow (https://stackoverflow.com/questions/37061089/trouble-with-tensorflow-in-jupyter-notebook). After following the steps shown in the answer and changed python executable python, as you recommended, from "/Users/username/anaconda/bin/python" to just "/Users/username/anaconda/envs/virtualproject/bin/python", the jupyter notebook successfully imported tensorflow!

[danbri]

FWIW on OSX, this helped me: https://github.com/jupyter/jupyter/issues/245#issuecomment-286927371

[K539]

I had a similar issue. It could be that your Jupyter notebook is looking into a different path than where you have installed tensorflow. You want to make sure Jupyter opens the correct Python through the correct kernel.
One way to fix this is to install nb_conda by typing the following in your terminal:

conda install -c anaconda-nb-extensions nb_conda

You can then run Jupyter and when selecting a notebook from the New dropdown menu, make sure you select the Python that is connected to the kernel you want (kernel where you have all the libraries and dependencies such as tensorflow)

[Kanagachidambaresan]

Hi guys,
I get these errors when i try to make a object detection on Rpi, tensorflow, how to proceed further any ideas??

IOError: [Errno socket error] [Errno -3] Temporary failure in name resolution
09/03/2018 10:45:46 AM INFO: Shutdown kernel
09/03/2018 10:45:47 AM WARNING: Exiting with nonzero exit status

[gnestor]

@Kanagachidambaresan Have you read/tried @Giccuk's solution?

https://stackoverflow.com/questions/37061089/trouble-with-tensorflow-in-jupyter-notebook

[DSLituiev]

I solved the problem by adding "env":{"LD_LIBRARY_PATH":"/usr/local/cuda-9.0/lib64:[...]" to my envirnment's kernelspec. As mentioned above, the spec directory can be found with jupyter kernelspec list

[mmattklaus]

I am on Windows 10 and using Tensorflow (tf) 1.13.
I had this problem as well. I had tf installed (using conda) into a conda virtual env (v-env).
I installed ipykernel and made it available within the notebook. However, I couldn't import tf from my notebook.

Here is how I resolved it.

• I cloned the virtual env (in which tf was installed) into a new virtual env (since I didn't want to download the entire package again).
• And it worked (I don't know why)

Summary:
- Create a conda virtual env
- Install tensorflow (into the v-env) using conda
- Install ipykernel into the v-env and install it. Help here

[Defcon27]

Try running the following in both the notebook and ipython:

import sys
sys.executable

Odds are they are using different python executables. If this is the case, you can update your python kernelspec. See #397 for reference.

Yes! that worked thanks

[ympaik87]

I have been trying to fix this problem for days, but still failing. Can anybody enlighten me what else I need to do?

My python environment is:

• windows 10
• python 3.7.7
• anaconda3 2020.2
• conda 4.8.2
• jupyter 1.0.0
• ipython 7.13.0
• ipykernel 5.1.4
• tensorflow 2.1.0
• ... (basically try to install latest versions)

What I have tried:

1. recreate conda env + add nb_conda
2. clean kernels in kernelspec & make sure python path point to the correct my virtual environment in kernel.json
3. change python path in kernel.json as following:

{
 "argv": [
  "python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3",
 "language": "python"
}

However, I'm still getting this:

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-3d3a52651c8b> in <module>
----> 1 import tensorflow as tf

~\scoop\apps\anaconda3\current\envs\kg37\lib\site-packages\tensorflow\__init__.py in <module>
     99 
    100 # We still need all the names that are toplevel on tensorflow_core
--> 101 from tensorflow_core import *
    102 
    103 # These should not be visible in the main tf module.

~\scoop\apps\anaconda3\current\envs\kg37\lib\site-packages\tensorflow_core\__init__.py in <module>
     38 import sys as _sys
     39 
---> 40 from tensorflow.python.tools import module_util as _module_util
     41 
     42 from . _api.v2 import audio

ModuleNotFoundError: No module named 'tensorflow.python.tools'; 'tensorflow.python' is not a package

If I try import sys; sys.executable, then I can import tensorflow as tf, but I have another error below:

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-2-fc4c73cf54bb> in <module>
----> 1 import tensorflow.keras.layers

ModuleNotFoundError: No module named 'tensorflow.keras.layers'; 'tensorflow.keras' is not a package

Both import tensorflow and import tensorflow.keras.layers work fine in terminal (in python as well as ipython), so I believe there is something wrong in jupyter environment, but I can't identify them.

Can anybody help me this issue???

---

I figured it out myself. I'm leaving the way how it got fixed, so someone might need to try this.
I just moved the notebook what I'm trying to run to the root path.
The relative path of the notebook before was something like samples/forcasting/covid-rnn.ipynb, and as soon as I moved this to ./covid-rnn.ipynb, the issue magically was gone.
I know some of the issues that people had caused by file names like csv.py, which was the same name as python package.
In my case, when tensorflow try to import submodules, somehow my notebook's relative path mess with it. It still doesn't make sense for me that other packages work just fine except the ones with tensorflow.

[dinis-rodrigues]

I had the same problem, as I was inside my conda environment. The jupyter wasn't using the respective python path. I followed some of the advices here and respective links. I present my solution.

By following this guideline:

If you want to have multiple IPython kernels for different virtualenvs or conda environments, you will need to specify unique names for the kernelspecs.

Make sure you have ipykernel installed in your environment.
conda install ipykernel # or pip install ipykernel

For example, using conda environments, install a Python (myenv) Kernel in a first environment:

source activate myenv
python -m ipykernel install --user --name myenv --display-name "Python (myenv)"

After this, restart/launch jupyter, open any notebook, and under the 'Kernel' tab you can specify in 'Change kernel' your newly created kernel.

Hope this helps, because it solved my issue