Keras: How to train LSTM with multiple sequences

[bangrui]

Hi, I have a question about training a LSTM with multiple time series. Could you tell me how does the BPTT work at this case? Suppose these sequences are related to each other, so I can't train each sequence separately. Also, there is no obvious order of these sequences so you can't train them using a multi-dimensional LSTM, as discussed in Alex Graves' paper.

[fchollet]

I believe nobody is answering you because nobody can understand what you are trying to say. Can you clarify?

[bangrui]

Sorry for the confusion. Let me clarify my questions. Suppose you have one stock, your target value is the stock price and you have 10 different features for this stock, like volatility, 5 day average etc. Then using LSTM and BPTT, I can build a recurrent neural network which works perfectly. However, if now I have two stocks, I believe I can still use LSTM as mentioned in https://github.com/fchollet/keras/issues/347. However, will the input(output) of stock 1 has influence for the stock 2? How does the BPTT work in this case? Is training two stocks at the same time equivalent to training them separately?

[fchollet]

If your input is (stock, time, features), then each stock will be treated independently. Yes, training on several samples at the same time is equivalent to separate training.

[bangrui]

@fchollet Thanks! That's very helpful. But do you know is there an LSTM algorithms will train them together? I think in Alex Grave's paper, he talked about the multi dimensional RNN, http://arxiv.org/pdf/0705.2011v1.pdf. However, if your sequences are stock, you can't put them in to a multi dimension space (at least there is no natural way to do it based on my knowledge).

[jfsantos]

If one stock affects the other, wouldn't it make more sense to group all stocks you're interested into into a single feature set (say, the concatenation of the 10 features for each stock) and train the LSTM with sequences of these concatenated features?

[jfsantos]

(btw, this is not a Keras issue, so I'm not sure the issue tracker is the best place for asking this kind of question since there is a mailing list)

[bangrui]

@jfsantos Thanks for your reply. Concatenation is a great idea but it will also increase the number of neurons in each layer a lot. I will look around in the mailing list see whether there is a similar question. Close the issue.