Probability: AttributeError: 'tuple' object has no attribute 'layer'

[cserpell]

I have been working with some sampling models, and I want to try the flexibility provided by tensorflow probability, specially regarding flows. I am struggling to create a basic working example, so I am here asking for help. I am playing with a forecasting network model, having an LSTM outputting mean and variance of a prediction. I used DistributionLambda to create a sampling layer at the end, but when creating the model, it fails with the following error. It seems to me that there is an incompatibility of keras and tensorflow versions, but I am using the most recent of both.

Here it is a minimim example notebook with the error.

Libraries versions:

Keras==2.2.4
tensorflow==1.14.0
tensorflow-probability==0.7.0

I appreciate any help on how to use the DistributionLambda layer in the context of a keras model.

Traceback (most recent call last):
  File "main.py", line 327, in <module>
    main()
  File "main.py", line 323, in main
    Script(args).run()
  File "main.py", line 158, in run
    the_models = self._train_model(x_train, y_train)
  File "main.py", line 116, in _train_model
    number_models=self._args.ensemble_number_models)
  File "main.py", line 106, in _train_models
    file_model='{}_{}'.format(file_model, num)))
  File "main.py", line 93, in _train_one_model
    x_val=x_val, y_val=y_val)
  File "/home/cserpell/git/inf475/project/modules/evaluate_model.py", line 62, in train
    validation_y=(None if y_val is None else y_val))
  File "/home/cserpell/git/inf475/project/models/model.py", line 192, in fit
    validation_y=validation_y)
  File "/home/cserpell/git/inf475/project/models/neural_net.py", line 139, in _internal_fit
    last_layer = self._create_net(input_layer)
  File "/home/cserpell/git/inf475/project/models/paper_lstm_sampled.py", line 73, in _create_net
    self._output_distribution)(last_layer)  # , training=True)
  File "/home/cserpell/git/inf475/project/p3/lib/python3.7/site-packages/tensorflow_probability/python/layers/distribution_layer.py", line 206, in __call__
    inputs, *args, **kwargs)
  File "/home/cserpell/git/inf475/project/p3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 663, in __call__
    inputs, outputs, args, kwargs)
  File "/home/cserpell/git/inf475/project/p3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1708, in _set_connectivity_metadata_
    input_tensors=inputs, output_tensors=outputs, arguments=kwargs)
  File "/home/cserpell/git/inf475/project/p3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1795, in _add_inbound_node
    input_tensors)
  File "/home/cserpell/git/inf475/project/p3/lib/python3.7/site-packages/tensorflow/python/util/nest.py", line 515, in map_structure
    structure[0], [func(*x) for x in entries],
  File "/home/cserpell/git/inf475/project/p3/lib/python3.7/site-packages/tensorflow/python/util/nest.py", line 515, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "/home/cserpell/git/inf475/project/p3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1794, in <lambda>
    inbound_layers = nest.map_structure(lambda t: t._keras_history.layer,
AttributeError: 'tuple' object has no attribute 'layer'

[cserpell]

Copying the main part of it here for clarity:

def create_net(input_shape):
    input_layer = layers.Input(shape=input_shape[1:])
    last_layer = input_layer
    # Input shape is (samples, steps, lags)
    last_layer = layers.LSTM(10)(last_layer)
    # LSTM output is already flatten here
    # Output dense layer with correct number of outputs.
    last_layer = layers.Dense(2)(last_layer)
    # Reshape output to put mean and variance first
    last_layer = layers.Reshape((2, 1, 1))(last_layer)
    last_layer = tfp.layers.DistributionLambda(
        make_distribution_fn=lambda t: tfp.distributions.Normal(loc=t[:, 0],
                                                                scale=tf.sqrt(tf.exp(t[:, 1]))),
        convert_to_tensor_fn=lambda s: s.sample())(last_layer)
    model = models.Model(inputs=input_layer, outputs=last_layer)
    # TODO: Change loss below to include variational loss
    model.compile(loss=losses.mean_squared_error, optimizer=optimizers.adam(lr=0.001))
    print(model.summary())

model = create_net(x_train.shape)
model.fit(x=x_train, y=y_train, validation_data=(x_val, y_val), epochs=200, batch_size=128)

[shubhaminnani]

did you got the solution @cserpell

[cserpell]

Yes, I forgot to mention. It was a libraries version issue. I used keras bundled with tensorflow and it worked fine.

[shubhaminnani]

Which version did you installed @cserpell

[SleepProgger]

I ran into the same problem and using tensorflow.keras solved it.
tensorflow.keras.__version__ reports 2.2.4-tf while keras>=2.2.2 does NOT work.

[Taichi-Pink]

Hi,@SleepProgger. Could you explain more for me? Did you change the version of Keras API in TensorFlow(tf.keras) from 2.2.4 to <2.2.2? If so, how did you change it? I'm using pycharm IDE, it just requires me to import tensorflow-gpu==1.14.0, but I don't know how to change the version of tf.keras.

[rao208]

Hi, I am getting the same error.
**print(tf.__version__)**:

2.0

**print(tf.keras.__version__)**

2.2.4-tf

I have tensorflow-gpu installed. I am getting this error when I run the following code:

import tf_explain
from tf_explain.callbacks.activations_visualization import ActivationsVisualizationCallback

callbacks = [
    ActivationsVisualizationCallback(validation_data=(x_val, y_val),
                                     layers_name=["hidden_layer_1"],
        output_dir="/home/raov/Desktop/Share/uni_freiburg/Datasets",
    ),
]


model.fit(x= x_train, y = y_train, batch_size=2, epochs=2, verbose= 1, callbacks=callbacks, validation_data= (x_val, y_val))

I guess it has something to do with the callback function. The source of the above code is https://github.com/sicara/tf-explain

Please note, that I tried model.fit with and without the validation data. Both the times I get the same error, so it has nothing to do with the validation data I guess

Any idea how to solve this problem?

[shubhaminnani]

Hi, this is version conflict of keras and tensorflow. Try the code with tensorflow==1.14 @rao208

[rao208]

@shubhaminnani Is there any other way? I need to use Tensorflow 2.0.

[amichayfeld]

I tried to downgrade Keras to 2.24 and check that both Keras and tf.Keras are 2.2.4, and still got this issue.

Edit:
i pass this issue by import tf.keras instead of import keras

[secezar]

Had the similar problem when tried to use custom LSTM layer written in tf, with keras Embedding layer. Changing
from keras.layers import Embedding
to
from tensorflow.keras.layers import Embedding
solved the problem.
tensorflow==2.0.0 from sources, tested both with Keras==2.2.4 and Keras==2.3.1

[rao208]

@secezar I agree. I also did the same and it works fine now :)

[Ankita-Das]

@secezar I did exactly as you said with, with tensorflow version='2.1.0-rc0' and keras version='2.3.1' but, it still shows the error

[grofte]

@secezar I did exactly as you said with, with tensorflow version='2.1.0-rc0' and keras version='2.3.1' but, it still shows the error

You aren't supposed to install Keras separately when you use tf2. Use the Keras bundled with the Tensorflow you have installed.
If it still doesn't work try disabling eager execution.

[SungmanHong]

ALSO in my TF 2.1.0!

[E008001]

AttributeError: 'tuple' object has no attribute 'layer'

(tf.keras.__version__)
2.2.4-tf
(tf.__version__)
2.0.0