Keras: Keras LSTM units

Created on 11 Nov 2017  路  2Comments  路  Source: keras-team/keras

The idea of this post is to get a deeper understanding of the LSTM argument "units". I don't know if it makes any difference but I am using Theano.
https://keras.io/layers/recurrent/
On this page, under "LSTM", units are explained as:

  • units: Positive integer, dimensionality of the output space.

Here some example lines of code just so that we have something specific that we can talk about:
model.add(LSTM(32, batch_size=50, input_shape(1,12))
model.add(Dense(5, activation='softmax')

I drew a schematic of how I understand the structure:
lstm_schematic

And now a picture from that page: http://colah.github.io/posts/2015-08-Understanding-LSTMs/
LSTM

In my terminology according to the picture of http://colah.github.io/posts/2015-08-Understanding-LSTMs/ "A" is a LSTM Block.
Input: x_t
Output of the hidden layer: h_t

And here are my questions:
Is it correct that model.add(LSTM(32)) means that there are 32 nodes in the hidden layer?
Are these 32 nodes equivalent to 32 LSTM Blocks?
Is it correct to assume that in one timestep the Input x_t goes through 32 LSTM blocks?
x_1 --> 32 LSTM blocks (input x_1 goes 32 times through a structure like "A")
x_2 --> 32 LSTM blocks (input x_2 goes 32 times through a structure like "A")
x_3 --> ...
Or does the number of units (32) has nothing to do with the number of LSTM blocks?
Is it possible that there is just one LSTM Block per timestep like in the picture from http://colah.github.io/posts/2015-08-Understanding-LSTMs/ and 32 means that the internal cell state c_t is a vector with 32 rows that passes the LSTM block ("A") just once?

Or do I mix something up here?

I hope I could make myself clear. If you have any questions don't hesitate to ask me.

I would appreciate any help :)

Most helpful comment

I found my answers here: https://github.com/fchollet/keras/issues/7600

And here are my questions:
Is it correct that model.add(LSTM(32)) means that there are 32 nodes in the hidden layer?

Yes.

Are these 32 nodes equivalent to 32 LSTM Blocks?

The answer here is definitely no.

Is it correct to assume that in one timestep the Input x_t goes through 32 LSTM blocks?
x_1 --> 32 LSTM blocks (input x_1 goes 32 times through a structure like "A")
x_2 --> 32 LSTM blocks (input x_2 goes 32 times through a structure like "A")
x_3 --> ...

No.

Or does the number of units (32) has nothing to do with the number of LSTM blocks?

Yes.

Is it possible that there is just one LSTM Block per timestep like in the picture from http://colah.github.io/posts/2015-08-Understanding-LSTMs/ and 32 means that the internal cell state c_t is a vector with 32 rows that passes the LSTM block ("A") just once?

Yes.

Or do I mix something up here?

Yes, I did. I mixed "LSTM block" and "units".

All 2 comments

I found my answers here: https://github.com/fchollet/keras/issues/7600

And here are my questions:
Is it correct that model.add(LSTM(32)) means that there are 32 nodes in the hidden layer?

Yes.

Are these 32 nodes equivalent to 32 LSTM Blocks?

The answer here is definitely no.

Is it correct to assume that in one timestep the Input x_t goes through 32 LSTM blocks?
x_1 --> 32 LSTM blocks (input x_1 goes 32 times through a structure like "A")
x_2 --> 32 LSTM blocks (input x_2 goes 32 times through a structure like "A")
x_3 --> ...

No.

Or does the number of units (32) has nothing to do with the number of LSTM blocks?

Yes.

Is it possible that there is just one LSTM Block per timestep like in the picture from http://colah.github.io/posts/2015-08-Understanding-LSTMs/ and 32 means that the internal cell state c_t is a vector with 32 rows that passes the LSTM block ("A") just once?

Yes.

Or do I mix something up here?

Yes, I did. I mixed "LSTM block" and "units".

  1. its not pretty clear...what is the use of 32.....what does it control?

what is the difference of nodes vs blocks?

2.what controls the number of LSTM block? is it elements of batch size which is equal to number of LSTM blocks>

Was this page helpful?
0 / 5 - 0 ratings