keras lstm cells

[vinayakumarr]

model.add(lstm(4)). This has 4 lstm blocks, but how many cells keras creates it? if I use stacked lstm, then keras lstm contain 2 hidden layer: it will be many to many. Is there any specific architecture keras has for lstm and as well as stacked lstm?

[carlthome]

model.add(LSTM(4)) creates four LSTM cells in a single layer. If you want to stack more layers, call model.add(LSTM(4)) more times (once per layer).

[vinayakumarr]

creates 4 lstm blocks and each block has 4cells
Is this right. Or it has 4 blocks where each block has a single cell?

[ishank26]

@vinayakumarr a "block" is a single cell or memory unit in reference to your query. As mentioned by @carlthome model.add(LSTM(4)) will create 4 cells or "blocks" in a single layer. For more info on lstm 1 & 2

[vinayakumarr]

according to thesis, first link lstm has blocks, which inturn has cells. But what about in keras? Reference fig 3.3 in thesis

[carlthome]

Hochreiter's thesis? It's in German, right? Feel free to translate and include the figure you're referencing.

[vinayakumarr]

no, FELIX GERS thesis, it is in english 1

[carlthome]

Keras' LSTM() assumes you want as many "memory blocks" as cells. If you want to share gates for several cell states you'll need to implement your own custom layer.

[carlthome]

So to answer your original question: it has 4 blocks where each block has a single cell.

[vinayakumarr]

yes you are right. so keras has 4 blocks where each block has a single cell for lstm(4) . in hidden layer 4 blocks which in turn has 4 cells: this has input gate, output gate, forget gate, error carousel (CEC) and peehole connection, fig 5.1 in thesis. Could you give a basic architecture for these 4 blocks. This helps to understand more clearly

one more question, in staked lstm, i will have two hidden layer am i right?

one hidden layer has 4 blocks , this layer feeds to other hidden layer (contains 4 blocks) for 2 stacked lastm layers am i right?

[carlthome]

Honestly, if you're that curious about the particular details of Keras's LSTM, just look at it? Here it is.

[ishank26]

In my opinion, before implementing a model you should take a deeper look into theory of RNNs and its variants. That'll will solve most of the problems. For cell architecture refer this and for stacked layer representation #1029