Keras: Problem when using timeDistributed with Convolution1D layer

[Wenhao-Jin]

I'm going to use TimeDistributed layer wrapper to combine CNN with LSTM layers. I just followed the link here: https://github.com/jamesmf/mnistCRNN, but using Convolution1D instead of Convolution2D layers. But I got an error when adding the TimeDistributed wrapped Convolution layers on the Embedding layers (shown below). Could you help me to check what is wrong here? Thanks!

model_aa = Sequential() model_aa.add(Embedding(input_dim=1000, output_dim=100, input_length=maxlen, dropout=0.2)) model_aa.add(TimeDistributed(Convolution1D(nb_filter=20, filter_length=5, border_mode='valid', activation='relu', subsample_length=1, init='glorot_normal'))) model_aa.add(TimeDistributed(MaxPooling1D(pool_length=3, border_mode='valid'))) model_aa.add(TimeDistributed(Convolution1D(nb_filter=10, filter_length=5, border_mode='valid', activation='relu', subsample_length=1, init='glorot_normal'))) model_aa.add(TimeDistributed(MaxPooling1D(pool_length=3, border_mode='valid'))) model_aa.add(TimeDistributed(Convolution1D(nb_filter=10, filter_length=3, border_mode='valid', activation='relu', subsample_length=1, init='glorot_normal'))) model_aa.add(TimeDistributed(MaxPooling1D(pool_length=3, border_mode='valid'))) model_aa.add(TimeDistributed(Convolution1D(nb_filter=10, filter_length=3, border_mode='valid', activation='relu', subsample_length=1, init='glorot_normal'))) model_aa.add(Bidirectional(LSTM(15, dropout_W=0.2, dropout_U=0.2, init='glorot_normal')))

IndexError Traceback (most recent call last)
in ()
1 model_aa = Sequential()
2 model_aa.add(Embedding(input_dim=1000, output_dim=100, input_length=maxlen, dropout=0.2))
----> 3 model_aa.add(TimeDistributed(Convolution1D(nb_filter=20, filter_length=5, border_mode='valid', activation='relu', subsample_length=1, init='glorot_normal')))
4 model_aa.add(TimeDistributed(MaxPooling1D(pool_length=3, border_mode='valid')))
5 model_aa.add(TimeDistributed(Convolution1D(nb_filter=10, filter_length=5, border_mode='valid', activation='relu', subsample_length=1, init='glorot_normal')))

/home/wjin/anaconda/envs/wjin/lib/python2.7/site-packages/keras/models.pyc
in add(self, layer)
310 output_shapes=[self.outputs[0]._keras_shape])
311 else:
--> 312 output_tensor = layer(self.outputs[0])
313 if type(output_tensor) is list:
314 raise Exception('All layers in a Sequential model '

/home/wjin/anaconda/envs/wjin/lib/python2.7/site-packages/keras/engine/topology.pyc
in __call__(self, x, mask)
485 'layer.build(batch_input_shape)')
486 if len(input_shapes) == 1:
--> 487 self.build(input_shapes[0])
488 else:
489 self.build(input_shapes)

/home/wjin/anaconda/envs/wjin/lib/python2.7/site-packages/keras/layers/wrappers.pyc in build(self, input_shape)
96 child_input_shape = (input_shape[0],) + input_shape[2:]
97 if not self.layer.built:
---> 98 self.layer.build(child_input_shape)
99 self.layer.built = True
100 super(TimeDistributed, self).build()

/home/wjin/anaconda/envs/wjin/lib/python2.7/site-packages/keras/layers/convolutional.pyc in build(self, input_shape)
113
114 def build(self, input_shape):
--> 115 input_dim = input_shape[2]
116 self.W_shape = (self.filter_length, 1, input_dim, self.nb_filter)
117 self.W = self.init(self.W_shape, name='{}_W'.format(self.name))

IndexError: tuple index out of range

Please make sure that the boxes below are checked before you submit your issue. Thank you!

• [x] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

• [ ] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

• [ ] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

[mercev]

I have the same issue. Did you solve it @Wenhao-Jin?

[marekzidek]

I have the same problem, even if I try to do it like in the example, with Conv2D and shape (sequence_len,1). Always getting list index out of range. Using tf backend.

[Rhuax]

Same here

[iamjli]

Ditto

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[chrisdinant]

Me too.

[GnarZiheng]

Same here. Has this issue been closed?

[Krekelmans]

same...

[jdhurwitz]

same

[alake1]

Any solutions?

[chenyikuan110]

Nobody is trying to answer this question. So sad.

[thomasmooon]

Same and still unsolved.

[thomasmooon]

In my case I solved that via reshaping the input dimension. As described here in the keras api, the input dimension of the 1D-Conv layer must be (batch_size, steps, input_dim). To be able to wrap it with the timeDistributed Layer, one needs to add a dimension, cause the tensor of the combination of timeDistributed(Conv1D(...)) needs to have the shape (batch_size,sequence,steps,input_dim). Hence one needs to reshape the tensor from 3D to 4D.
This behaviour is implicitly described in the examples of the timeDistributed documentation.

Example:
My input tensor (batch-size, steps, input_dim) has values (?, 12, 1000):

• 1000 features
• 12 observations per feature = 12 timepoints

The goals is to apply a 1D convolution over the 12 timepoints with weights shared between features (hence, time-distributed).

Approach:
Reshape the (?, 12,1000) tensor to (?, 1000, 12, 1) tensor.
After applying a 1D-Conv with 3 Kernels of width 5 the output tensor has shape (?, 1000, 8, 3).
More precisely given a np.array X with shape e. g. (1189, 12, 1000) one retrives a tensor of shape (1189,1000,12,1) with X.reshape((-1, 1000, 12, 1))

[vonjackustc]

Me too.

[mosew]

Have you tried changing 'valid' to 'same'?