Keras: How to understand input data of LSTM layer in Keras?

[Yingyingzhang15]

I'm new to Keras, and I find it hard to understand the shape of input data of the LSTM layer.The Keras Document says that the input data should be 3D tensor with shape (nb_samples, timesteps, input_dim).

I understand in this way ,please tell me if I'm wrong. If I have 1000 sentences ,each sentence has 10 words, and each word is presented in a 3-dim vector, so 1000 is nb_samples, 10 is the timesteps and 3 is the input_dim, am I right?

Another question, if the shape of my data is 9000* 10 , 9000 is the nb_sample, and 10 is the input_dim, should I set the time step as 1? If so ,how could the LSTM find time information during training? Actually, I want to give every five 1*10 vectors a label, should I set the timesteps as 5 and reshape my data?

Thank you very much!

[Yingyingzhang15]

@EderSantana Can you answer my questions?Thank you!

[NasenSpray]

I understand in this way ,please tell me if I'm wrong. If I have 1000 sentences ,each sentence has 10 words, and each word is presented in a 3-dim vector, so 1000 is nb_samples, 10 is the timesteps and 3 is the input_dim, am I right?

yep

Another question, if the shape of my data is 9000* 10 , 9000 is the nb_sample, and 10 is the input_dim, should I set the time step as 1? If so ,how could the LSTM find time information during training? Actually, I want to give every five 1*10 vectors a label, should I set the timesteps as 5 and reshape my data?

yep, you have to reshape

[Yingyingzhang15]

@NasenSpray Thank you very much!

[Sandy4321]

Thanks
On Mar 23, 2016 04:33, "Yingyingzhang15" [email protected] wrote:

@NasenSpray https://github.com/NasenSpray Thank you very much!

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/fchollet/keras/issues/2045#issuecomment-200245976

[jeffreynghm]

@Yingyingzhang15 thanks for the example and question, i am also struggling with the similar question you have. but I have already solved my problem with input_shape...but still have the error for the past week...

My Case:
data_bin shape:(80733, 5) 80733 = nb_sample, 5 = no. of column per sample
Output for each sample is the one-hot vector (with 14 unique values), with 80733 sample i.e. (80733,14)...though I have make my code very simple: 1d input with 1d output...how come it still does not work?

Error message:

  File "/M01_AutomatedTradeMaker_v0.21.py", line 84, in baseline_tradeOps
    model.add(LSTM(100,input_shape = (None,1,1)))
  File "/usr/local/lib/python3.5/dist-packages/keras/models.py", line 292, in add
    layer.create_input_layer(batch_input_shape, input_dtype)
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 374, in create_input_layer
    self(x)
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 474, in __call__
    self.assert_input_compatibility(x)
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 415, in assert_input_compatibility
    str(K.ndim(x)))
Exception: Input 0 is incompatible with layer lstm_1: expected ndim=3, found ndim=4

My code:

# create model
        model = Sequential()
        model.add(LSTM(100,input_shape = (None,1,1)))
        model.add(Activation('softmax'))
        model.add(Dense(output_dim=1, activation='sigmoid'))
        print('Model loaded.')
        model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
        print('Model compiled.')
        return model

[HengLin1994]

@jeffreynghm maybe you can try like this one:
data = data.reshape(len(data),1,10)

model.add(LSTM(100, batch_input_shape=(BATCH_SIZE, 1, num),return_sequences=True,stateful=True,init='normal'))

model.add(layers.Activation('relu'))

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[gatsby97]

model=Sequential()
model.add(LSTM(50,return_sequences=True,input_shape=(x_train.shape[0],x_train.shape[1])))
model.add(BatchNormalization(epsilon=1e-06, momentum=0.9, weights=None))
model.add(Flatten())
model.add(Dense(units=1,activation='softmax'))
model.compile(loss='mae',optimizer='adam',metrics=['accuracy'])
model.fit(x_train,y_train,epochs=10,batch_size=10)
Error message
Traceback (most recent call last):
File "", line 1, in
File "/root/anaconda3/lib/python3.6/site-packages/keras/models.py", line 867, in fit
initial_epoch=initial_epoch)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 1522, in fit
batch_size=batch_size)
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 1378, in _standardize_user_data
exception_prefix='input')
File "/root/anaconda3/lib/python3.6/site-packages/keras/engine/training.py", line 132, in _standardize_input_data
str(array.shape))
ValueError: Error when checking input: expected lstm_1_input to have 3 dimensions, but got array with shape (580419, 23)

[hxd1011]

@Yingyingzhang15 if you just have 1 sample in the sequence, why use RNN?

[tobiaspiechowiak]

Exactly my question... but that mean you go back one sample, right?

lstm(100) gives the output layer of 100 hidden units, right?