Keras: Storing hidden stated of Recurrent layer

[rajendramishra18]

Does save_weights() in model also save the hidden cells and states of LSTM layer. I have to use inner cells of LSTM as a prior to another LSTM. As far as I understand we can store weights of LSTM and load it. But what about inner cell and hidden states?

[NasenSpray]

Nope, but you can get/set them manually:

import keras.backend as K

def get_states(model):
    return [K.get_value(s) for s,_ in model.state_updates]

def set_states(model, states):
    for (d,_), s in zip(model.state_updates, states):
        K.set_value(d, s)

[tdihp]

@NasenSpray Is the model in your code a Sequential object? I found my model.state_updates to be an empty list. However I can access states using model.layers[0].states.

my model object is:

batch_size = 32
seq_steps = 128

model = Sequential()
model.add(
    SimpleRNN(32,
            return_sequences=True,
            batch_input_shape=(batch_size, seq_steps, 1),
            stateful=True,
            )
)
model.add(TimeDistributed(Dense(1)))
model.compile(loss='mse', optimizer='rmsprop')

[NasenSpray]

@tdihp: yes, but the code is also from before Keras' big API overhaul, so not sure if it works with the current version.

[oetcoleman]

@tdihp @NasenSpray - has anyone tried this on a recent version of Keras? I'd like to implement something similar now, so would be interested if the code above still works.

[shlomi-schwartz]

@NasenSpray Could you please have a look at https://stackoverflow.com/questions/54850854/keras-restore-lstm-hidden-state-for-a-specific-time-stamp?noredirect=1#comment96476550_54850854 and say if your solution applies there too?