Keras: Need advisory about interconnected layers and models (Autoencoder)

[lkaupp]

Just done this guide -> https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html and the model saved successfully. Now i want to separate training phase from prediction. I know its possible to load a model again using load_model() function. In this tutorial you got on lstm for encoding, one for decoding, how to get the encoder/decoder again after successfully load the model?

In addition how to solve such a problem:

/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py:2344: UserWarning: Layer lstm_2 was passed non-serializable keyword arguments: {'initial_state': [<tf.Tensor 'lstm_1/while/Exit_2:0' shape=(?, 366) dtype=float32>, <tf.Tensor 'lstm_1/while/Exit_3:0' shape=(?, 366) dtype=float32>]}. They will not be included in the serialized model (and thus will be missing at deserialization time).

So far i know it is necessary to have these weights/connection between encoder and decoder. Any idea?

[lkaupp]

First i named my lstm layers, then i tried the following, with bad results:

trainedModel = load_model('s2s.h5')

'# Next: inference mode (sampling).
'# Here's the drill:
'# 1) encode input and retrieve initial decoder state
'# 2) run one step of decoder with this initial stateencoder_inputs = Input(shape=(10, 183))

decoder_inputs = Input(shape=(None, 183))
encoder_inputs = Input(shape=(10, 183))
encoder = trainedModel.get_layer("encoder")
encoder_outputs, state_h, state_c = encoder(encoder_inputs)

'# We discard encoder_outputs and only keep the states.

encoder_states = [state_h, state_c]

'# and a "start of sequence" token as target.
'# Output will be the next target token
'# 3) Repeat with the current target token and current states

decoder_lstm = trainedModel.get_layer("decoder")
decoder_outputs, _, _ = decoder_lstm(decoder_inputs,
initial_state=encoder_states)
decoder_dense = Dense(183, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)`

followed by this error:

Traceback (most recent call last): File "predict.py", line 70, in <module> newlines = decode_sequence(chunks_train[2]) File "predict.py", line 54, in decode_sequence predictedLogline, h, c = decoder_model.predict(states_value) File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 1730, in predict check_batch_axis=False) File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 89, in _standardize_input_data '...') ValueError: Error when checking model : the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 3 array(s), but instead got the following list of 2 arrays: [array([[ 4.84920815e-02, -2.58697212e-01, 3.05500686e-01, 3.09119525e-04, -4.98499274e-02, -4.02525216e-01, -9.96050686e-02, -2.76536420e-02, 3.88025880e-01, -5.6...

After all, i tried something else, loading all layers from model:

`trainedModel = load_model('s2s.h5')

'# Next: inference mode (sampling).
'# Here's the drill:
'# 1) encode input and retrieve initial decoder state
'# 2) run one step of decoder with this initial

decoder_inputs = trainedModel.get_layer('input_2')
encoder_inputs = trainedModel.get_layer('input_1')
encoder = trainedModel.get_layer("encoder")
encoder_outputs, state_h, state_c = encoder(encoder_inputs)

'# We discard encoder_outputs and only keep the states.

encoder_states = [state_h, state_c]

'# and a "start of sequence" token as target.
'# Output will be the next target token
'# 3) Repeat with the current target token and current states

decoder_lstm = trainedModel.get_layer("decoder")
decoder_outputs, _, _ = decoder_lstm(decoder_inputs,
initial_state=encoder_states)
decoder_dense = trainedModel.get_layer('dense_1')
decoder_outputs = decoder_dense(decoder_outputs)`

Covered with this error:

File "predict.py", line 20, in <module> encoder_outputs, state_h, state_c = encoder(encoder_inputs) File "/usr/local/lib/python3.5/dist-packages/keras/layers/recurrent.py", line 483, in __call__ return super(RNN, self).__call__(inputs, **kwargs) File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 587, in __call__ self.assert_input_compatibility(inputs) File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 432, in assert_input_compatibility str(inputs) + '. All inputs to the layer ' ValueError: Layer encoder was called with an input that isn't a symbolic tensor. Received type: <class 'keras.engine.topology.InputLayer'>. Full input: [<keras.engine.topology.InputLayer object at 0x7f528618fc50>]. All inputs to the layer should be tensors.

Any ideas, whats could be wrong with my code?

[lkaupp]

Found my mistake, forgot to call the prediction with an empty array that can be filled. Reverted to my first approach.

[tcl326]

@CloudResearch did the Serialization Warning give you any problems later on?

[lkaupp]

Well i can't say that at the moment. i could predict something, but i don't know how it affects my prediction. Would be nice to know, what is not saved. As far as i understand the message output dimension of lstm1 won't get stored. So i decided to just load the layer from the model and connect all layer again. idk if that's a solution or if it slows down the process at all.

[Woekiki]

Hey guys, having the same problem! The fix mentioned above doesn't work with me. I'm using Keras 2.1.4 and implementing the same autoencoder pattern as described above. However loading my model gives completely different results. I hope a bugfix will arive soon as for now I can't use my work

[DariusAlexander]

Got the same problem, too.

[wimthiels]

me too..

[LeZhengThu]

I believe it's the callbacks that raise the non-serializable warning, but have no idea on how to fix it.

[cibeah]

Hey ! I'm encountering the same problem too !
Can anybody here correctly save and load weights from their model after the "non-serializable keyword arguments" warning ?
I'm running on Keras 2.1.5 and every time I try loading my saved weights I get completely different results. This makes it impossible to resume training or start predictions after any kernel interruption.
I've tried model.save(), model.save_weights(), callbacks to a checkpointer with and without "save_weights_only=True", also tried to pickle the weights after .get_weights(), but I just can't get around it. .save_weights() does not give any warning, but it still doesn't save my weights correctly.

[guidesc]

Hi all,

I have the same issue here. Could someone reopen the issue? Thanks for any advice for suggestions?

[piacenti]

Same problem for me. It seems that I can load it for inference but if I call the fit method then it seems to start from scratch, actually it starts making slower progress than if it was from scratch.

[joewandy]

Is there any resolution to this issue or hints on how to fix it? Could someone reopen the issue?

@CloudResearch did you manage to fix the serialisation problem?

[cibeah]

Hey !

Coming back here because I think I have a way around. It's not really pretty, but it works for me.

To save the model I get the weights list with model.get_weights() and serialize them on my disk. Then to reload them, I set the weights manually layer by layer. I've been able to resume training with this method.

My function to reload the weights :

def reload_weights(model, weights_list):
    k=0
    for l in model.layers:
        weights = l.get_weights()
        n = len(weights)
        if n > 0:
            l.set_weights(weights_list[k:k+n])
            k+=n
    print("Poids chargés")

From this it should be easy to create a new callback that works, although I haven't had the time to try it yet.

[martvl]

@cibeah Have you had the chance to try and resume training or start predicting with your method? Currently having the same issues still in Keras 2.2.0. Did you pickle the weights_list to disk to reload it after?

Edit:
This is the error in 2.2.0:
/usr/local/lib/python3.6/dist-packages/keras/engine/network.py:888: UserWarning: Layer lstm_2 was passed non-serializable keyword arguments: {'initial_state': [<tf.Tensor 'lstm_1/while/Exit_2:0' shape=(?, 256) dtype=float32>, <tf.Tensor 'lstm_1/while/Exit_3:0' shape=(?, 256) dtype=float32>]}. They will not be included in the serialized model (and thus will be missing at deserialization time).

[piacenti]

@cibeah I tried doing the same you did and it didn't work. I even tried doing the same in a different way and it still didn't work. I even verified that the weights before and after saving matched (using both your method and mine) but it still performed poorly. I can only guess that something that should be getting saved is not.

[KingDavidOfTepper]

The error actually comes from model.get_weights() - so attempts to save the weights and reload them using custom functions will also fail.

[wishvivek]

So, is there any solution to the Non-serialisation problem yet? Anyone?

[victorE23]

Got the same problem. Any solution??

[chenqiuyuan]

The same problem, ask for solutions or workaround.