Keras: Error installing tensorflow/keras with install_keras()

Created on 3 Oct 2017 · 24Comments · Source: rstudio/keras

So, as in the new version I see there is no need for devtools, I just run:

>install.packages("keras")
>library(keras)
>install_keras()

(Obviously I do not even try to install GPU version of tf)
Then I get this output:

> install_keras()
Creating r-tensorflow conda environment for TensorFlow installation...
Fetching package metadata .............
Solving package specifications: .

Package plan for installation in environment D:\Anaconda3\envs\r-tensorflow:

The following NEW packages will be INSTALLED:

    asn1crypto:      0.22.0-py36h8e79faa_1     
    ca-certificates: 2017.08.26-h94faf87_0     
    cachecontrol:    0.12.3-py36hfe50d7b_0     
    certifi:         2017.7.27.1-py36h043bc9e_0
    cffi:            1.10.0-py36hae3d1b5_1     
    chardet:         3.0.4-py36h420ce6e_1      
    colorama:        0.3.9-py36h029ae33_0      
    cryptography:    2.0.3-py36h123decb_1      
    distlib:         0.2.5-py36h51371be_0      
    html5lib:        0.999999999-py36ha09b1f3_0
    idna:            2.6-py36h148d497_1        
    lockfile:        0.12.2-py36h0468280_0     
    msgpack-python:  0.4.8-py36h58b1e9d_0      
    openssl:         1.0.2l-vc14hcac20b0_2      [vc14]
    packaging:       16.8-py36ha0986f6_1       
    pip:             9.0.1-py36hadba87b_3      
    progress:        1.3-py36hbeca8d3_0        
    pycparser:       2.18-py36hd053e01_1       
    pyopenssl:       17.2.0-py36h15ca2fc_0     
    pyparsing:       2.2.0-py36h785a196_1      
    pysocks:         1.6.7-py36h698d350_1      
    python:          3.6.2-h09676a0_15         
    requests:        2.18.4-py36h4371aae_1     
    setuptools:      36.5.0-py36h65f9e6e_0     
    six:             1.10.0-py36h2c0fdd8_1     
    urllib3:         1.22-py36h276f60a_0       
    vc:              14-h2379b0c_1             
    vs2015_runtime:  14.0.25123-hd4c4e62_1     
    webencodings:    0.5.1-py36h67c50ae_1      
    wheel:           0.29.0-py36h6ce6cde_1     
    win_inet_pton:   1.0.1-py36he67d7fd_1      
    wincertstore:    0.2-py36h7fe50ca_0        

#
# To activate this environment, use:
# > activate r-tensorflow
#
# To deactivate an active environment, use:
# > deactivate
#
# * for power-users using bash, you must source
#

Determining latest release of TensorFlow...done
Installing TensorFlow...
Traceback (most recent call last):
  File "D:\Anaconda3\envs\r-tensorflow\Scripts\pip-script.py", line 6, in <module>
    from pip import main
  File "D:\Anaconda3\envs\r-tensorflow\lib\site-packages\pip\__init__.py", line 28, in <module>
    from pip.vcs import git, mercurial, subversion, bazaar  # noqa
  File "D:\Anaconda3\envs\r-tensorflow\lib\site-packages\pip\vcs\subversion.py", line 9, in <module>
    from pip.index import Link
  File "D:\Anaconda3\envs\r-tensorflow\lib\site-packages\pip\index.py", line 33, in <module>
    from pip._vendor import html5lib, requests, six
ImportError: cannot import name 'html5lib'
Error: Error 1 occurred installing packages into conda environment r-tensorflow
In addition: Warning message:
running command '"D:/Anaconda3/Scripts/activate" r-tensorflow && pip install --upgrade --ignore-installed "https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-1.3.1-cp36-cp36m-win_amd64.whl"' had status 1

Source

andreidi

Most helpful comment

@h0111in: So, here is basically what you should do (for no GPU tf ver):

1. Fresh install Anaconda
2. conda create --name r-tensorflow python=3.5
3. activate r-tensorflow
4. pip install --ignore-installed --upgrade tensorflow 
5. conda install -c conda-forge keras

Basically if you do this you dont need to install_keras() at all !
Just install keras package and it will automaticly use the r-tensorflow env...

andreidi on 12 Oct 2017

👍16 🎉6

All 24 comments

It looks to me like your install.packages ended up installing keras 2.0.6 (one version older than the current one on CRAN). This may be because the CRAN build of keras 2.0.8 just went live in the last 24 hours and your mirror didn't quite have it yet. Try re-installing Keras with:

install.packages("keras", type = "source")

Then:

library(keras)
install_keras()

jjallaire on 3 Oct 2017

I understand but this should not be a show-stopper ... As it can be seen is not much related to keras version ....

andreidi on 3 Oct 2017

I don't understand the nature of these Anaconda html5lib issues. It appears as if TensorFlow is depending on an older version of html5lib but that's not squaring with the use of pip in new versions of Anaconda. See these discussions:

https://stackoverflow.com/questions/46499808/pip-throws-typeerror-parse-got-an-unexpected-keyword-argument-transport-enco

https://github.com/tensorflow/tensorboard/issues/588

I'd appreciate any help or insight those more knowledgable of pip and conda that I might have.

jjallaire on 3 Oct 2017

Yes, any help would be greatly appreciated.

As a work-around I created manually from command prompt the r-tensorflow env and installed tf, keras and all required packages.

andreidi on 4 Oct 2017

Another user having a similar problem uninstalled and then re-installed Anaconda and the problem went away: https://github.com/rstudio/keras/issues/146

jjallaire on 4 Oct 2017

It seems the tensorflow-gpu installation installs an old version of the html5lib package that corrupts the usage of the pip command. Any subsequent pip command after the first attempt to install tensorflow-gpu will give this error.

You need to upgrade html5lib after the error appears. Open an Anaconda prompt window and type:
"activate r-tensorflow" (on a mac put the word source in front) ,then force it to update to .999999999 from .9999999 with : "conda update -f html5lib"

it is probably also possible to do from within R environment but this is one way. hope it helps.

oxijas on 7 Oct 2017

Could the forced update of html5lib cause subsequent problems with tensorflow?

jjallaire on 7 Oct 2017

Could the forced update of html5lib cause subsequent problems with tensorflow?
it might yes.. as it breaks the 'bleach' package apparently. https://github.com/tensorflow/tensorboard/pull/604
at this point I am not sure if there is any other solution

oxijas on 8 Oct 2017

@andreidi What version of Anaconda are you running? (you can find out via reticulate::conda_version())

@brandondbutcher was running Anaconda 4.3.27 and then made the problem go away via updating to newer version of Anaconda. Interestingly I am running Anaconda 4.3.23 and don't see the problem. The current release of Anaconda as of Sept 26th is Anaconda 5.0.0.

I have a suspicion that this issue is isolated to certain versions of Anaconda because I know we are getting hundreds of installs per-day and I have only 2 reports of the problem.

jjallaire on 10 Oct 2017

@jjallaire I installed 5.0.0.0.
Actually, like I previously said, I manually created& prepared the r-tensorflow env forcing tf version at 1.3.0 and it worked. Found out that there is a bug in Anaconda: see https://github.com/conda/conda/issues/6079

andreidi on 10 Oct 2017

@andreidi , I have the same issue. would you please explain how you 'created and prepared the r-tensorflow env forcing tf ...'.
I re-installed Anaconda several times, upgraded and downgraded html5lib via Anaconda navigator and/or Anaconda prompt, however, none of them were helpful.

h0111in on 11 Oct 2017

@h0111in: So, here is basically what you should do (for no GPU tf ver):

1. Fresh install Anaconda
2. conda create --name r-tensorflow python=3.5
3. activate r-tensorflow
4. pip install --ignore-installed --upgrade tensorflow 
5. conda install -c conda-forge keras

Basically if you do this you dont need to install_keras() at all !
Just install keras package and it will automaticly use the r-tensorflow env...

andreidi on 12 Oct 2017

👍16 🎉6

@andreidi , Great! it works.
Appreciated.

h0111in on 12 Oct 2017

I have a fix or sorts available here: https://github.com/rstudio/tensorflow/commit/6cb685ae5434d7ddf2ebf731d31508c17cdf8805

The fix is essentially to avoid using pip whenever possible and to instead install from conda-forge. Unfortunately pip is required for the GPU version as well as for nightly builds (or any build that references a package URL rather than version).

I think that the issue hasn't been seen that widely yet because it is triggered by Anaconda 5.0.0. It seems quite likely that the underlying issue(s) will be addressed within TensorBoard soon (since they result in the breaking of pip for all conda environments on a system). Hopefully there is also something that Anaconda can do to have better isolation of pip dependencies.

I'm going to keep the issue open as a reminder to check on the status of the TensorBoard and Anaconda issues whose interaction causes these problems:

https://github.com/tensorflow/tensorboard/issues/588

https://github.com/conda/conda/issues/6079

jjallaire on 13 Oct 2017

https://github.com/SerpentAI/SerpentAI/issues/96#issuecomment-336203943

BenjaminWegener on 2 Dec 2017

Hi @jjallaire and @andreidi , I followed your steps but had no luck.
I am on a linux machine with load balanced rstudio server. My storage space is shared between all the load balanced nodes and we use a shared package library. The system is RedHat 7 and I installed anaconda 4.2 , python 3.5 using the anaconda .sh file. The system has default python 2.7 which I did not remove.

Also setting up proxy for all users is prohibited. So using install_keras() for all users is not an option.
I keep getting Error: Unable to find conda binary. Is Anaconda installed? when I tried use_condaenv() or other such functions.

reticulate::py_config() (output below) shows that RStudio system still sees Python 2.7.5 and not Python 3.5

python:         /usr/bin/python
libpython:      /usr/lib64/python2.7/config/libpython2.7.so
pythonhome:     /usr:/usr
version:        2.7.5 (default, Aug  2 2016, 04:20:16)  [GCC 4.8.5 20150623 (Red Hat 4.8.5-4)]
numpy:          /usr/lib64/python2.7/site-packages/numpy
numpy_version:  1.7.1
keras:          [NOT FOUND]

Could this be a problem and if not, any other thoughts what else could I be missing ?

tarunparmar on 11 May 2018

You can use conda install -c hesi_m keras to install the latest version of Keras (2.1.6) and Tensorflow(1.8) for you.

VanitarNordic on 28 May 2018

@andreidi On October 11, 2017 you state "1. Fresh install Anaconda" I'm having trouble doing the fresh install on Win10 (amd-x64). Which release of Anaconda did you use? Thx in advance. Suggestions welcome.

woodbomb50 on 8 Aug 2018

@woodbomb50

Create a fresh Anaconda environment and use conda install -c hesi_m keras which installs Keras 2.2.2 and Tensorflow 1.9.0 for you.

VanitarNordic on 8 Aug 2018

Thanks, but WHICH anaconda environment?

On Wed, Aug 8, 2018 at 10:58 AM Hesam Moshiri notifications@github.com
wrote:

@woodbomb50 https://github.com/woodbomb50

Create a fresh Anaconda environment and use conda install -c hesi_m keras
which installs Keras 2.2.2 and Tensorflow 1.9.0 for you.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/rstudio/keras/issues/147#issuecomment-411436569, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABftRzPkILt8nika8WEMnKfAwltmWLxoks5uOvyUgaJpZM4Prpsg
.

woodbomb50 on 8 Aug 2018

@woodbomb50

Create an Anaconda environment by command:

conda create -n your_desired_name python=3.6.6

Then activate it by

activate your_desired_name

Look here: https://conda.io/docs/user-guide/tasks/manage-environments.html

VanitarNordic on 8 Aug 2018

Thankyou. Which version of keras have you installed and, if so, on which version of R?

woodbomb50 on 8 Aug 2018

Might be related to #730.

marioem on 7 Apr 2019

@h0111in: So, here is basically what you should do (for no GPU tf ver):
1. Fresh install Anaconda
2. conda create --name r-tensorflow python=3.5
3. activate r-tensorflow
4. pip install --ignore-installed --upgrade tensorflow 
5. conda install -c conda-forge keras
Basically if you do this you dont need to install_keras() at all !
Just install keras package and it will automaticly use the r-tensorflow env...

This works very well. If I want a gpu version, what should step 5 be? (in step 4 use tensorflow-gpu in place of tensorflow)

TIA!