Keras: Can we define each time step of a RNN with different length?

Created on 20 Sep 2015  路  3Comments  路  Source: keras-team/keras

After using LSTM, I thought each of the time steps in a RNN input has identical length.
For example, I have an input vector I with length n.
Thus,
I[0:3] belong to time step 1;
I[3:6] belong to time step 2;
etc.

However, what if I want each time step has different length?
Like
I[0:4] belong to time step 1;
I[4:6] belong to time step 2;
I[6:15] belong to time step 3;
etc.

Should I write a new RNN layer to implement such function?

stale

Most helpful comment

Unclear what you mean.

If your samples have different numbers of timesteps, you can resort to padding or masking (see the Masking layer for instance).

If your samples have different numbers of features, you can simply fill in the missing features with zeros or some other fixed "filler" value. You could also resort to having binary categorical features that denote whether a given feature is present or not.

All 3 comments

Can you elaborate on this? It sounds like you have a different number of features at each timestep. Most models are not capable of this; you need a constant number of features. What do I[0:4], I[4:6], ... in your example correspond to?

If you are talking about using a different number of timesteps per sequence then that is possible but is a slight hassle to deal with.

Unclear what you mean.

If your samples have different numbers of timesteps, you can resort to padding or masking (see the Masking layer for instance).

If your samples have different numbers of features, you can simply fill in the missing features with zeros or some other fixed "filler" value. You could also resort to having binary categorical features that denote whether a given feature is present or not.

@colincsl @fchollet Thank you for your responding.
yes, I can re-model my problem more clearly.
Input: a vector, just thinking, each element of this vector represents a "timestep".
Note: Thus, all my samples have the same numbers of "timesteps".
Output: The values in several "timesteps".
That means, I need to project the values of some certain timesteps, while most of the timesteps in the input should be "masked out".
The precondition is that I don't know the exact interval between any two of the projection timesteps.

Probably, it is not a real "RNN" problem, isn't it?
I am thinking to define a new layer. This layer projects the "timesteps" with the prob.>0.8 (0.8 is a threshold which can be tuned in the future). Such operation may solve the problem.

Do you think so? Thank you a lot!

Was this page helpful?
0 / 5 - 0 ratings