Keras: Any way to use LSTM with Dynamic Timesteps?

[hqsiswiliam]

How can I use LSTM layer with dynamic timesteps so that I could feed data like below:

[[data],[data],[data]],
[[data],[data],[data],[data]],
[[data],[data],[data],[data],[data]],
[[data],[data]]
...
]

Layer I used：
model = Sequential() model.add(LSTM(32,input_length=nb_timesteps,input_dim=input_dim,return_sequences=False) model.add(Dropout(0.5)) model.add(Dense(input_dim =32, output_dim = 1))

[farizrahman4u]

• Pad sequences, or do bucketing
• Don't specify input_length

[hqsiswiliam]

@farizrahman4u Hi, I found pad sequences but it add too much zeros if the data have huge diversity in length. Then, I tried to removeinput_length from model and I got following message:
Exception: Error when checking model input: expected lstm_input_1 to have 3 dimensions, but got array with shape (4545, 1)

[phquang]

Try to do something like

model = Sequential()
model.add(LSTM(32, input_shape = (None, your_input_dim),  return_sequences = False)

In this case, each input your network takes have shape of (batch_size, time_steps, your_input_dim) where your_input_dim is the dimension of your input at each time step, time_steps is the length of the sequence and batch_size is the number of sequences in each batch.
If you use batch_size = 1, i.e., you feed only one sequence at a time, then each sequence can have any length. If you use batch_size > 1, then those sequences in that batch must be padded to have the same length

[hqsiswiliam]

@QuangHongPham Hi, I specify input shape to (None, input_dim). However, when I print out model.layers[0].input_shape it returns me (None, None, 5) and cannot accept my sequence.
Here is my code, correct me if I am wrong:

``````
for X,Y in zip(X_train,y_train):

hist = model.fit(X, Y,nb_epoch=param['nb_epoch'], batch_size=1,

    validation_split=param.get('validation_split'), verbose=0 \

    , callbacks=[checkpointer,history]
    )```

``````

[fchollet]

I found pad sequences but it add too much zeros if the data have huge diversity in length.

Bucket your data by sequence length.

[phquang]

@hqsiswiliam You may need to check the shape of your training data, here is an example code and it runs just fine

model = Sequential()
model =  model.add(LSTM(32, input_shape = (None, 50), return_sequences = False))
# create one training sequence of length 7
x = np.random.randn(1,7,50)
# create a random label
y = np.random.randn(1,32)
# fit the model
model.fit(x,y) 

[farizrahman4u]

Bucket your data by sequence length.

@fchollet
While bucketing one has to make sure that there is no hidden correlation between sequence length and the sequence meaning.. else the batches will not be random enough.

[hqsiswiliam]

@farizrahman4u @fchollet @QuangHongPham Thanks for your reply. I end up with keras.preprocessing.sequence.pad_sequences and it work well than I expected. Thank you all.

[LeZhengThu]

@fchollet @farizrahman4u Are there any special handling steps for Keras to accept a None timestep? In Keras code recurrent.py line 571, the code is "timesteps = input_shape[1]". If I don't specify a timestep value, then input_shape[1]=None and python would raise an error.