Keras: I get AttributeError on tf 0.10.0 running model_from_json

[kgtekito]

cnn = model_from_json(jsn)
  File "/usr/local/lib/python3.5/dist-packages/keras/models.py", line 197, in model_fr
om_json
    return layer_from_config(config, custom_objects=custom_objects)
  File "/usr/local/lib/python3.5/dist-packages/keras/utils/layer_utils.py", line 36, i
n layer_from_config
    return layer_class.from_config(config['config'])
  File "/usr/local/lib/python3.5/dist-packages/keras/models.py", line 1025, in from_co
nfig
    model.add(layer)
  File "/usr/local/lib/python3.5/dist-packages/keras/models.py", line 308, in add
    output_tensor = layer(self.outputs[0])
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 515, in __call__
    self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 573, in add_inbound_node
    Node.create_node(self, inbound_layers, node_indices, tensor_indices)
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 150, in create_node
    output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
  File "/usr/local/lib/python3.5/dist-packages/keras/layers/core.py", line 91, in call
    x = K.in_train_phase(K.dropout(x, self.p, noise_shape), x)
  File "/usr/local/lib/python3.5/dist-packages/keras/backend/tensorflow_backend.py", line 1243, in in_train_phase
    x = tf.python.control_flow_ops.cond(tf.cast(_LEARNING_PHASE, 'bool'),
AttributeError: module 'tensorflow.python' has no attribute 'control_flow_ops'

The issue is reproduced by doing a model_from_json on tf 0.10.0, this doesnt happen below tf 0.9.0. Thanks.

[tripiana]

Good luck I have a similar issue:

from keras.models import Sequential
from keras.layers.core import Flatten, Dense, Dropout
from keras.layers.convolutional import Convolution2D, MaxPooling2D,
ZeroPadding2D
from keras.optimizers import SGD

model = Sequential()
model.add(ZeroPadding2D((1, 1), input_shape=(3, 224, 224)))
model.add(Convolution2D(64, 3, 3, activation='relu'))
model.add(ZeroPadding2D((1, 1)))
model.add(Convolution2D(64, 3, 3, activation='relu'))
model.add(MaxPooling2D((2, 2), strides=(2, 2)))

... (etc)

model.add(Flatten())
model.add(Dense(4096, activation='relu'))
model.add(Dropout(0.5))

and then:

AttributeError: module 'tensorflow.python' has no attribute
'control_flow_ops'

if I use add(Dropout(1)) it works. Just for using a float value...

[fchollet]

TF 0.10 appears to be breaking some imports. I'll look into it, we have to fix it while maintaining compatibility with 0.9.

if I use add(Dropout(1)) it works.

Of course it does, because 1 just disables dropout.

[brunodoamaral]

I had the same problem with master version of TF today. Changed to TF 0.10.0 rc0 solved the issue for me.

[tripiana]

@fchollet is there any advances on this? I'm afraid, but we are stuck in a scenario where we need tf 0.10 (not rc0) because it solves some buf that affect us, and we are working with Keras and affected by this bug (no attribute control_flow_ops). A bit of help will he much appreciades. Thanks in advance.

[baregawi]

I think I found the issue.

I'm running Ubuntu 14.04.5 LTS with tensorflow==0.10.0 and keras==1.1.0. I received the same error as above from keras/backend/tensorflow_backend.py

AttributeError: module 'tensorflow.python' has no attribute 'control_flow_ops'

I attempted to install TF 0.10.0 rc0 but the installation failed for unrelated reasons (though, in hindsight, it would have avoided the root problem if the installation did work).

After some digging around, I found out that tensorflow had made a change to remove undocumented symbols in their tensorflow/python/__init__.py: https://github.com/tensorflow/tensorflow/commit/5563229b5b45917cd8155bc47a8356af90fb20a2#diff-6ad579dc34adb7a581b8c0bcb1a4dd79

Probably as an accident, they made it so that control_flow_ops was no longer visible as a submodule of tensorflow.python. But it can still be imported with from tensorflow.python.ops import control_flow_ops so I added from tensorflow.python.ops import control_flow_ops as tf_control_flow_ops at the top of keras/backend/tensorflow_backend.py and replaced all references to tensorflow.python.control_flow_ops with tf_control_flow_ops and it fixed the issue.

I'll go over the contribution guide and submit a pull request.

[baregawi]

I've opened an issue with tensorflow: https://github.com/tensorflow/tensorflow/issues/4616
I'll wait to see if they are planning on fixing it on their side before submitting a pull request.

[vrv]

Hi, I think we can change to just using the 'public' access point via tf.cond, no need to descend into the private modules. Same for any of the control flow ops! It is too easy in python to expose private details such as internal module structure, so we're tightening up the API so that most uses go through the proper public namespace.

@fchollet we should probably figure out how to switch all keras use to the public TF interface in a way that's appropriately backwards compatible for keras.

[tripiana]

Hey guys. The TF issue was closed so, is there any update on this? I think it's not fixed yet. I think they expect to have it fixed by keras.

[al626]

Quick fix / hack workaround for v0.11

import tensorflow as tf
tf.python.control_flow_ops = tf

model = keras.models.Sequential()
# ...

[al626]

Ok, looks like the following parts of the TF private API are used:

:~/git/keras$ grep -n "tensorflow\.python" keras/backend/tensorflow_backend.py
2:from tensorflow.python.ops import control_flow_ops
3:from tensorflow.python.training import moving_averages
5:    from tensorflow.python.ops import ctc_ops as ctc
1164:        from tensorflow.python.ops.rnn import _dynamic_rnn_loop
1786:    from tensorflow.python.ops import functional_ops

It doesn't seem like too much work to get this working using the public API, so I'm going to have a go at it unless someone else is working on this?

So far:

• control_flow_ops is just a syntax change.
• moving_averages only calls assign_moving_average, which isn't accessible under the public API yet.
• ctc_ops is a syntax change.
• _dynamic_rnn_loop is a bigger change, tf.nn.rnn_cell.RNNCell needs subclassing and a lot of the rnn function needs rewriting.
• syntax change for functional_ops

Also note that tests/keras/backend/test_backends.py::TestBackend::test_rnn_no_states is failing under TF 0.11 due to this:

:~/git/keras$ grep -n -A 1 "use dummy state" keras/backend/tensorflow_backend.py
1172:            # use dummy state, otherwise _dynamic_rnn_loop breaks
1173-            state = inputs[:, 0, :]

state is being set to dimension sequence_length * input_size whereas it needs to be batch_size * output_size. This probably didn't crash before because TF didn't do any shape checking, but it seems to me that this would give buggy output?

What are the requirements for backwards compatibility for Keras with TF?

[al626]

I have a version that's a little hacky but passing the tests for TF 0.11 and 0.10 here but it's failing spectacularly on TF 0.9 (on things that seem completely unrelated...).

I'm away until 12th October so will try to finish it up and submit a pull request then, but I would greatly appreciate it if anyone has any advice on things I could do better or why the everything's failing on 0.9! @fchollet or @vrv maybe?

Thanks

[tripiana]

With a master version of keras and a master version of tensorflow right now this issue is closed.

[lesshaste]

On Thu 13 Oct 2016 I did

pip install keras --user
pip install tensorflow --user

and then https://github.com/fchollet/keras/blob/master/examples/mnist_cnn.py gives me

AttributeError: 'module' object has no attribute 'control_flow_ops'

Will the pypi repositories be updated with working versions soon?

print tensorflow.__version__
0.11.0rc0
print keras.__version__
1.1.0

It also doesn't seem to be possible to switch to an older version of tensorflow.

pip install tensorflow==
Collecting tensorflow==
Could not find a version that satisfies the requirement tensorflow== (from versions: 0.11.0rc0)
No matching distribution found for tensorflow==

[kumarneeraj2005]

I was getting following error while running kears and tensor flow as backend

x = tf.python.control_flow_ops.cond(tf.cast(_LEARNING_PHASE, 'bool'),
AttributeError: 'module' object has no attribute 'control_flow_ops'

but when i did following, error got vanished, now its working fine
import tensorflow as tf
tf.python.control_flow_ops = tf

[vrv]

If you are using any version of TF since 0.8, you can and should just use tf.cond. Since https://github.com/fchollet/keras/commit/240fd5b68e033ee31167581fcd061dc027282fcf Keras's access to tf.cond has been fixed.

[spMohanty]

you can simply go for this monkey patch in your code :

import tensorflow
from tensorflow.python.ops import control_flow_ops 
tensorflow.python.control_flow_ops = control_flow_ops

[sloth2012]

aha! It works for me! @spMohanty