Keras: `import keras` with tensorflow>=1.2.0 on windows is very slow

[syagev]

Running import keras on Python 3.5.2 on Windows with tensorflow version 1.2.0 and above takes over 10 seconds (both tensorflow and tensorflow-gpu).

Downgrading to tensorflow 1.1.0 reduces the import command time to less than 3 seconds.

Profiling the import keras method I see a huge bottleneck due to the following import tree:

-> keras.activations
  -> keras.engine
    -> keras.engine.training
      -> keras.callbacks
        -> tensorflow.contrib

With the import tensorflow.contrib statement taking up 67% inclusive time. Digging deeper than that is a bit tricky since the bottlenecks are not so significant any more.

A follow up question is why are we importing so many modules as a default, and whether a leaner import can be enabled (e.g. import only stuff required for inference without training). I'm asking since running from keras.models import Sequential is enough to trigger the entire import tree above and still take >10 seconds.

On my linux box import times are more acceptable (<5 seconds) with any tf version.

• [x] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

• [x] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

• [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

[patrikerdes]

I can reproduce this issue on macOS, it takes 10 seconds for me to do from keras.models import Sequential.

Tensorflow version: 1.2.1
Keras version: master branch
Python version: 3.6.2
macOS version: Sierra (10.12.5)

Script used to reproduce the issue:

import time
before = time.time()
from keras.models import Sequential
print(time.time() - before)

[mrTsjolder]

I happened to be using keras version 2.0.2 where the import went well (under 5s), but after updating to 2.0.6, the script of @patrikerdes took over 15 seconds.

Tensorflow version: 1.2.1
Keras version: 2.0.6
Python version: 3.5.2
macOs version: Sierra 10.12.6

[jayantj]

I'm facing a similar issue on Ubuntu (using 14.04 currently):

# python3.5.2

# Keras==2.0.6, tensorflow==1.1.0
%time import keras
Wall time: 2.04 s

# Keras==2.0.2, tensorflow==1.1.0
%time import keras
Wall time: 1.42 s

# Keras==2.0.6, tensorflow==1.2.1
%time import keras
Wall time: 11.8 s

# Keras==2.0.2, tensorflow==1.2.1
%time import keras
Wall time: 1.34 s

[patrikerdes]

I found out why tensorflow.contrib is slow to import and submitted a PR to tensorflow, see https://github.com/tensorflow/tensorflow/issues/11829 and https://github.com/tensorflow/tensorflow/pull/11830
If this PR is accepted, the time to import keras would go from ~10 to ~3 seconds.

I also implemented a different fix in Keras, where tensorflow.contrib is imported in the __init__ method of the Tensorboard callback, since it is only needed if the Tensorboard callback is instantiated. https://github.com/patrikerdes/keras/commit/ef41d4b9265440857b5eb1f4a5d13e2a326b4e4f

If the Tensorflow PR is not accepted, I could created a PR of the Keras fix. If the Tensorflow PR is accepted, I'm not sure the Keras fix is necessary, it would then take the time to import keras from ~3 to ~2 seconds.

[gatapia]

Any update on this? looks like such a simple solution recommended by @patrikerdes.
I think waiting on TF to accept PRs is a mistake, should just fix in Keras, especially if the fix is that simple. @patrikerdes did you ever turn your commit into a keras PR?