Keras: Unable to load keras

[dnaeye]

I'm down to the last few hairs on my head after stressing over this for days. The problem is that I can't get keras to work well on my Macbook (Mojave OS v.10.14.2).

At first, I tried installing it from RStudio:

install.packages("keras")
library(keras)
install_keras()

But then, it couldn't find keras, which makes sense from the output of py_config():

python:         /usr/bin/python
libpython:      /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/libpython2.7.dylib
pythonhome:     /System/Library/Frameworks/Python.framework/Versions/2.7:/System/Library/Frameworks/Python.framework/Versions/2.7
version:        2.7.10 (default, Aug 17 2018, 19:45:58)  [GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.0.42)]
numpy:          /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy
numpy_version:  1.8.0
keras:          [NOT FOUND]

python versions found: 
 /Users/dhong/anaconda/envs/r-tensorflow/bin/python
 /usr/bin/python
 /usr/local/bin/python3
 /Users/dhong/anaconda/bin/python
 /Users/dhong/anaconda3/bin/python

Then I heard that changing the PATH variable in the .Renviron file would work:

Sys.getenv('PATH')
usethis::edit_r_environ()

In the file that would open, I added this line, based on the result from Sys.getenv('PATH'):
PATH=/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/opt/X11/bin

That seemed to have worked, but then when I tried to fit a keras model, I would run into a fatal error that would crash RStudio. This is the last message I would see before that happened:

I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA

The latest thing I've tried in Terminal is to try the solution suggested here:

conda create --name tensorflow python=3.5
activate tensorflow
conda install jupyter
conda install scipy
pip install tensorflow
# or
# pip install tensorflow-gpu

But now I get the following error message in RStudio when running library(keras):

Error: package or namespace load failed for ‘keras’:
 .onLoad failed in loadNamespace() for 'tensorflow', details:
  call: py_module_import(module, convert = convert)
  error: ImportError: No module named tensorflow

@jjallaire or anyone, please help! If it isn't obvious, I have no idea what I'm doing.

[matthiasmace]

What do you want to do ?
If coding within R, just quit and try "library(keras)' and this tutorial : https://keras.rstudio.com/.
If you want to access from outside R, you must get into the created virtualenv by typing 'workon R-tensorflow' and then launch python as usual.
Does this fix your problem ?

[dnaeye]

I'm trying to use keras within RStudio, which is why I even mentioned it. I'm guessing that my install issues are related to it being pointed to the wrong version of Python, but I don't know how to do this correctly. In any case, bottom line is I can't get keras working in RStudio.

[ChuckFinley]

I'm having the same issue. It seems the conda env created by install_keras has an issue. Starting with a clean conda, I tried:

keras::install_keras()
reticulate::use_condaenv("r-tensorflow")
reticulate::py_config()

and here's what I see:

python:         /usr/bin/python
libpython:      /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/libpython2.7.dylib
pythonhome:     /System/Library/Frameworks/Python.framework/Versions/2.7:/System/Library/Frameworks/Python.framework/Versions/2.7
version:        2.7.10 (default, Aug 17 2018, 19:45:58)  [GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.0.42)]
numpy:          /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy
numpy_version:  1.8.0

python versions found: 
 /Users/frank/anaconda3/envs/r-tensorflow/bin/python
 /usr/bin/python
 /usr/local/bin/python3
 /Users/frank/anaconda3/bin/python

So it's creating the conda env r-tensorflow but reticulate won't use it.

[skeydan]

@ChuckFinley what happens if after a restart of R, you immediately do

reticulate::use_condaenv("r-tensorflow")

and then,

reticulate::py_config()

and then, try to create a keras model?

[ChuckFinley]

@skeydan That doesn't work. But I think I've found part of the issue. After keras::install_keras I opened a terminal and tried to use python from the new env:

conda activate r-tensorflow
python

Within python:

import numpy

Gave me this error:

ImportError: Something is wrong with the numpy installation. While importing we detected an older version of numpy in ['/Users/frank/miniconda3/envs/r-tensorflow/lib/python3.6/site-packages/numpy']. One method of fixing this is to repeatedly uninstall numpy until none is found, then reinstall this version.

I tried

conda uninstall numpy
conda install numpy

But then in R I got

> reticulate::use_condaenv("r-tensorflow", require = TRUE)
> reticulate::py_config()
python:         /Users/frank/miniconda3/envs/r-tensorflow/bin/python
libpython:      /Users/frank/miniconda3/envs/r-tensorflow/lib/libpython3.6m.dylib
pythonhome:     /Users/frank/miniconda3/envs/r-tensorflow:/Users/frank/miniconda3/envs/r-tensorflow
version:        3.6.8 |Anaconda, Inc.| (default, Dec 29 2018, 19:04:46)  [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
numpy:           [NOT FOUND]

NOTE: Python version was forced by use_python function

[ChuckFinley]

From the install_keras output, it looks like numpy 1.14 and 1.16 are both installed, maybe?

1.14:

Successfully installed ... numpy-1.14.5

1.16:

The following NEW packages will be INSTALLED:

...

  numpy              conda-forge/osx-64::numpy-1.16.1-py36_blas_openblash486cb9f_0

In Terminal, conda is listing 1.14:

(r-tensorflow) Maxs-MacBook-Pro-3:~ frank$ conda list numpy
# packages in environment at /Users/frank/miniconda3/envs/r-tensorflow:
#
# Name                    Version                   Build  Channel
numpy                     1.14.5                   pypi_0    pypi

But the version of python in the site-package is 1.16

cat /Users/frank/miniconda3/envs/r-tensorflow/lib/python3.6/site-packages/numpy/version.py

# THIS FILE IS GENERATED FROM NUMPY SETUP.PY
#
# To compare versions robustly, use `numpy.lib.NumpyVersion`
short_version = '1.16.1'
version = '1.16.1'
full_version = '1.16.1'
git_revision = '685b9ace06f1dc50e2698099d7a2b6a241379318'
release = True

if not release:
    version = full_version

[skeydan]

Thanks, this is helpful!

Can you try installing version 1.12, like so:

install_tensorflow(version = "1.12")

[ChuckFinley]

@skeydan That worked! Thanks, you rock!

[skeydan]

Thanks to you for the informative error report!

For the background: We have been installing TF 1.10 because of a bug in that will only be fixed in TF 1.13 (which should be out anytime but unfortunately isn't).

In the unlikely case that you should run into that bug ("list index out of range"), please check if TF 1.13 is out in which case we'll be installing TF 1.13 by default, that is, a simple install_tensorflow should yield 1.13 then :-)

[skeydan]

Hello to everybody on this thread,

if you still have problems using install_tensorflow() from RStudio, please try again now.

First install tensorflow directly from GitHub, as in

devtools::install_github("rstudio/tensorflow")
devtools::install_github("rstudio/keras")

Then do

tensorflow::install_tensorflow()
tensorflow::tf_config()

which should give you version 1.12.

We have been installing TF 1.10 until yesterday because of a bug in that will only be fixed in TF 1.13 (which should be out anytime but unfortunately isn't yet). Now with people having installation problems due to incompatibilities with numpy as installed by conda, we decided to switch to TF 1.12 now and as soon as TF 1.13 is actually available, we'll install that by default.

Thanks for reporting the problem!

If you still run into problems installing, please open another issue and describe the problem, indicating the output of

reticulate::py_discover_config()
reticulate::use_condaenv("r-tensorflow")
reticulate::py_config()

Thanks!

[ghost]

Hi I have the same issue.

Still can't get keras to work on RStudio after trying out everything said on this thread.

```

reticulate::py_discover_config()
python: /usr/bin/python
libpython: /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/libpython2.7.dylib
pythonhome: /System/Library/Frameworks/Python.framework/Versions/2.7:/System/Library/Frameworks/Python.framework/Versions/2.7
version: 2.7.10 (default, Feb 22 2019, 21:55:15) [GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)]
numpy: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy
numpy_version: 1.8.0

python versions found:
/usr/bin/python
/usr/local/bin/python3
Warning messages:
1: In mget(objectNames, envir = namespace, inherits = TRUE) :
restarting interrupted promise evaluation
2: In (function (x, ...) : restarting interrupted promise evaluation
3: In get(method, envir = home) :
restarting interrupted promise evaluation

reticulate::use_condaenv("r-tensorflow")
reticulate::py_config()
python: /usr/bin/python
libpython: /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/libpython2.7.dylib
pythonhome: /System/Library/Frameworks/Python.framework/Versions/2.7:/System/Library/Frameworks/Python.framework/Versions/2.7
version: 2.7.10 (default, Feb 22 2019, 21:55:15) [GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)]
numpy: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy
numpy_version: 1.8.0
leidenalg: [NOT FOUND]

python versions found:
/usr/bin/python
/usr/local/bin/python3```