Keras: LSTM for time series prediction

[ashwinnaresh]

This is more of a conceptual clarification.

I have the following architecture :

model = Sequential()
model.add(LSTM(batch_size,
               batch_input_shape=(batch_size, tsteps, 1)))  
model.compile(loss='mse', optimizer='rmsprop',metrics=["accuracy"])

The LSTM is able to make pretty accurate predictions as of now.

Training data :

Note that the training data for each of the 3 weeks is almost similar in shape.

LSTM predictions on unseen testing data :

As you can see, the LSTM is still able to trace the unexpected peak in the graph which was never seen in the training data.

Any reason as to how the LSTM is able to do the above?

[gamehere007]

@ashwin123
What is your input and output data for this model

[hijoe320]

@ashwin123 I am sure there is one step lag between the actual time series and the predicted time series, this is the most seen "trap" if you do time series prediction like this, in which the NN will always mimic previous input of time series.

[sjayakum]

@hijoe320 yup you are right, there was a 1 time-step lag between predicted and actual. the NN was mimicking the previous input of the time series.
Thanks!

[sjayakum]

@hijoe320 Ok, So are we doing any thing wrong to end up with this?

The main problem we have is the "Testing Data" real-time we get only 1 time-step at a time. So they are tested on a batch-size = 1 and time-step = 1.

Anyways to overcome this "trap"? :grin:

---

More information

batch_size = 1
tsteps = 1

We have trained Each Timestep as a sample.

Say lstm requires input of the form => nb_samples x tsteps x attributes

where a,b,c,d... is a time series data
10:00 am -> X = 1 x 1 x a and y = b goes into lstm

10:01 am -> X = 1 x 1 x b and y = c goes into lstm..

10:02 am -> X = 1 x 1 x c and y = d goes into lstm.. And So on..

[hijoe320]

@sjayakum you can try:

1. randomize training samples in each batch, make sure they are not followed one by one
2. choose or design a better loss function other than MSE
3. extract some features from the input time series
4. manually limit the weight of x_{t-1}, x_{t-2}

[iwinterlovermy]

hi there. May I know how to use LSTM for multiple step ahead forecasting? Assuming I would like to predict 7 days ahead outcome. I've been trying few days but seems to get stuck - how should i prepare the data, and and what to set for batch size and time steps. Thanks in advance!

[safa94]

please did you find something to solve your problem i have the same problem here

[SunlnuS]

Hello, l'm using LSTM to predict time series and I meet the same problem. It seems that LSTM outputs input as output. Have you solved this problem? Would you like to give me some advice to solve this problem?

[yyb1995]

I have the same problem while using LSTM to predict a complex time series. There is a delay between true value and predicted value and they have a simular shape. It seems that the network copies the last input value to generate an output value. Are there any solutions to this problems? I will be very grateful.

[shafferjohn]

In my opinion, loss function comes to minimum, so the network chooses to mimic the actual value. You will get same result even without LSTM, I think, even just with one fully connect layer.

[kartik1225]

same problem any idea how to overcome this? i have detailed question on stackoverflow