Keras: How to train a many to one mapping sequential using LSTM

[lood339]

I have a problem like this:
It is a time sequential. At each time, the input is 14-dimension vector, the output is 1 dimension real value. So it is a regression problem. I want to train a fixed length sequence to sequence LSTM. So one input out example (a short sequnce) is X1, X2 ... Xt, the output is Y1, Y2 ...Yt. Note X1 is 14 dimension. I have many short sequences for training.
In the test, I have a fixed length sequence and want to output a result sequence.
How can I model is in keras, particularly how is the shape of the input training data (both features and labels)?
Thanks very much.

[fchollet]

Both input and output data should be 3D:
input: (batch_size, timesteps, 14)
output: (batch_size, timesteps, 1)

You just need to use a LSTM (or GRU, etc) layer with return_sequences=True, for instance:

model = Sequential()
model.add(LSTM(32, return_sequences=True, input_shape=(timesteps, 14))
model.add(TimeDistributedDense(1))

[lood339]

Thanks for your help. I have tested the model, and it is successfully compiled. A following question as I am confused by two "patch_size"
input: (batch_size, timesteps, 14). To my understanding, the "batch_size" is the number of the training data, right? For example, I have 1,000 training data, timesteps = 15 so the input is (1000, 50, 14)
In the model fitting, I write:
model.fit(X_train, y_train, batch_size = 16, verbose=1, nb_epoch=1)
The batch_size = 16 here has different meaning. It is the number of training data at each iteration, right? So the "batch_size" here can be different than the number of training data.

[farizrahman4u]

@lood339 Actually the size of your training data is (nb_samples, timesteps, 14) and (nb_samples, timesteps,1). nb_samples is some big number.. like 1000 as in your case. Your training data is split into batches, each batch of shape (batch_size, timesteps, 14) , (batch_size, timesteps, 1) where batch_size is some relatively smaller number like 16 or 32. And the model is trained one batch at a time. So be careful with the terms "training data" and "input data".

[talentlei]

@farizrahman4u thanks for your help! While how can I understand 'timesteps' ?

So one input out example (a short sequnce) is X1, X2 ... Xt, the output is Y1, Y2 ...Yt. Note X1 is 14 dimension.

Is the "timesteps" meaning the fixed length of LSTM when unfolding? like the t in the upon sample?

[farizrahman4u]

If x1, x2, x3, x4 ... xt is your sequence, then t is your timesteps.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[behdadahmadi]

I have a binary classification problem that makes me very confused about input,output of modeling with LSTM.
I want to input 5 rows of dataset ,and get the label color of 6th row.
My X has 5 features : rb , us, ls, Volume, pos
My Y is a label which is 1 or 0
My dataframe:

         rb        us        ls    Volume  pos  color
0  0.080071  1.000000  0.964286  0.020507    1      1
1  0.017798  0.857143  0.857143  0.017643    1      1
2  0.026698  0.642857  0.500000  0.031085    4      1
3  0.029666  0.833333  1.000000  0.011411    2      0
4  0.008899  0.500000  1.000000  0.008371    4      0

I make a sequence of 5 rows like this:

[[1.77976353e-02 8.57142857e-01 8.57142857e-01 1.76426968e-02
   1.00000000e+00]
  [2.66977791e-02 6.42857143e-01 5.00000000e-01 3.10845400e-02
   4.00000000e+00]
  [2.96664095e-02 8.33333333e-01 1.00000000e+00 1.14109866e-02
   2.00000000e+00]
  [8.89918602e-03 5.00000000e-01 1.00000000e+00 8.37062257e-03
   4.00000000e+00]
  [1.48316083e-02 8.33333333e-01 1.00000000e+00 8.47275749e-03
   2.00000000e+00]]

How I should classify Y of 6th row? I mean how to classify the 6th row OR first next item of next sequence? Do I need to declare it to the network somehow?
Is there any problem with my input/output of network? How can I get the label(Binary classification) of 6th item label color?
My code:

import pandas as pd
from Util import window_nd as Reshape
from keras.utils import to_categorical
from keras.layers import Dense,RNN,LSTM,Activation,Dropout
from keras.models import Sequential


df = pd.read_csv('./EURUSD_DATAFRAME.csv')
print(df.head())
df['pos'] = df['pos'].astype('int')
dat = df.values

X = dat[0:30004,0:5]
Y = dat[0:30000,5]
X = Reshape(X,5)  #It's a custom function which reshaped the X to serialized (5,5) 
#Shape of X is  (30000, 5, 5)

Y = to_categorical(Y,num_classes=2)

model = Sequential()
model.add(LSTM(25,input_shape=(5,5),return_sequences=True))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(LSTM(1))
model.add(Activation('relu'))
model.add(Dense(2))
model.add(Activation('sigmoid'))
model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])
model.fit(X,Y,batch_size=5,epochs=10,validation_split=0.2)
p = model.predict(X)
print(p)


js = model.to_json()
w = model.save_weights('bin11.h5')
with open('binModel11.json','w') as f:
    f.write(js)

I searched a lot. I also asked about it on StackOverflow.com ,but found no answer or I couldn't understand it.

[igorkf]

I have a binary classification problem that makes me very confused about input,output of modeling with LSTM.
I want to input 5 rows of dataset ,and get the label color of 6th row.
My X has 5 features : rb , us, ls, Volume, pos
My Y is a label which is 1 or 0
My dataframe:

         rb        us        ls    Volume  pos  color
0  0.080071  1.000000  0.964286  0.020507    1      1
1  0.017798  0.857143  0.857143  0.017643    1      1
2  0.026698  0.642857  0.500000  0.031085    4      1
3  0.029666  0.833333  1.000000  0.011411    2      0
4  0.008899  0.500000  1.000000  0.008371    4      0

I make a sequence of 5 rows like this:

[[1.77976353e-02 8.57142857e-01 8.57142857e-01 1.76426968e-02
   1.00000000e+00]
  [2.66977791e-02 6.42857143e-01 5.00000000e-01 3.10845400e-02
   4.00000000e+00]
  [2.96664095e-02 8.33333333e-01 1.00000000e+00 1.14109866e-02
   2.00000000e+00]
  [8.89918602e-03 5.00000000e-01 1.00000000e+00 8.37062257e-03
   4.00000000e+00]
  [1.48316083e-02 8.33333333e-01 1.00000000e+00 8.47275749e-03
   2.00000000e+00]]

How I should classify Y of 6th row? I mean how to classify the 6th row OR first next item of next sequence? Do I need to declare it to the network somehow?
Is there any problem with my input/output of network? How can I get the label(Binary classification) of 6th item label color?
My code:

import pandas as pd
from Util import window_nd as Reshape
from keras.utils import to_categorical
from keras.layers import Dense,RNN,LSTM,Activation,Dropout
from keras.models import Sequential


df = pd.read_csv('./EURUSD_DATAFRAME.csv')
print(df.head())
df['pos'] = df['pos'].astype('int')
dat = df.values

X = dat[0:30004,0:5]
Y = dat[0:30000,5]
X = Reshape(X,5)  #It's a custom function which reshaped the X to serialized (5,5) 
#Shape of X is  (30000, 5, 5)

Y = to_categorical(Y,num_classes=2)

model = Sequential()
model.add(LSTM(25,input_shape=(5,5),return_sequences=True))
model.add(Activation('relu'))
model.add(Dropout(0.2))
model.add(LSTM(1))
model.add(Activation('relu'))
model.add(Dense(2))
model.add(Activation('sigmoid'))
model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])
model.fit(X,Y,batch_size=5,epochs=10,validation_split=0.2)
p = model.predict(X)
print(p)


js = model.to_json()
w = model.save_weights('bin11.h5')
with open('binModel11.json','w') as f:
    f.write(js)

I searched a lot. I also asked about it on StackOverflow.com ,but found no answer or I couldn't understand it.

You said here: "I want to input 5 rows of dataset ,and get the label color of 6th row."
So I think you need to use 5 timesteps...this mean that you have to reshape your data to 3D, like LSTM needs:

[number_of_samples, timesteps, features]

As you have 5 timesteps and 5 features:

X = your_array.reshape(number_of_samples, 5, 5)

And so pass it to the LSTM layer:

....
model.add(LSTM(25, input_shape=(X.shape[1], X.shape[2]), return_sequences=True))
....