Keras: Error when checking model target

[thecolorblue]

I am running into an issue when putting my own data into the lstm example on the keras website. The only changes I mean to make are to fit the dataset I am working with into this example.

I created a gist here: https://gist.github.com/thecolorblue/5b672e706db17e744c935a4995747c1c

But to summarize I get this error when calling fit:

Exception: Error when checking model target: expected activation_3 to have shape (None, 1) but got array with shape (10, 11)

Here is the model:

model = Sequential()
model.add(Embedding(10, 32, input_length=11))
model.add(LSTM(32, activation='sigmoid', inner_activation='hard_sigmoid'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))

Please make sure that the boxes below are checked before you submit your issue. Thank you!

• [x] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps
• [x] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps
• [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

[fchollet]

Your targets are 11-dimensional and you are feeding them to an output that is 1-dimensional. That is your problem, and it is precisely what the error message describes.

[thecolorblue]

Where do I set the output dimension?

I tried changing the dense layer to model.add(Dense(11)) but I got the same error.

[thecolorblue]

If anyone finds themselves here later, the issue was not with the model but the shape of Y_train, which should have had a shape of (10,1). 10 to match the length of my X_train tensors first dimension, and 1 because we can only estimate one value for each column. Y_train can also be called classes, or target as it is here, depending on the context.

[ryubidragonfire]

Hi @thecolorblue , I am facing similar error, but still don't quite understand how to solve it.

For the final layer: model.add(Dense(10))
I assume it means it has 10 output classes.

Given that i have the following:
y_train.shape = (70, 1), consisting the labels of 10 classes, which looks like this:

y_train = 

array([5, 0, 3, 5, 6, 5, 4, 9, 7, 8, 6, 6, 3, 5, 5, 0, 5, 0, 4, 4, 5, 5, 4,
0, 5, 5, 4, 0, 6, 6, 5, 0, 5, 4, 7, 3, 5, 1, 5, 5, 3, 5, 8, 0, 5, 0,
5, 5, 5, 5, 6, 5, 4, 5, 5, 5, 2, 5, 4, 5, 9, 0, 0, 5, 9, 4, 6, 5, 0,
4], dtype=int64)

I am getting this error:
Exception: Error when checking model target: expected activation_50 to have shape (None, 10) but got array with shape (70, 1)
Could you please help me to understand:

1. Why shape (None, 10)is expected? What does that mean?
2. how to make it work for y_train that looks like above?

Thanking you in advance.

[billhsia]

i face the same problem with @ryubidragonfire , any body give tips ... thanks

[adwin5]

@ryubidragonfire
1, it expects (None,10) because your last layer having size 10, you should y_train=to_categorical(y_train, nb_classes=10) to change your training data from numerical to categorical one-hot vector
2,After convert, the shape should be (70,10) which is fit for (None, 10).
(None, 10) means the first dimension can be any size, but the second should be 10. In other words, you can have variable length of training data, but each data should just have size 10

[thecolorblue]

@adwin5, shouldn't the y_train values always have a shape of (len(X), 1)? If I understand correctly, the Y values should be the correct output corresponding to the X inputs. This is what the model should be training on. Giving a model (None, 10) shaped Y means you are expecting 10 different output values.

@ryubidragonfire & @billhsia, sorry, but I still don't know keras enough to say how to fix this problem. What I ended up doing is taking the same data and try to solve the problem using scikit-learn, which has better examples. Once you get the data in the right shape and understand the features you want to train on, move back over to keras. In my experience, there is more opportunity in improving accuracy by working on feature detection than in improving the model layers.

[adwin5]

@thecolorblue @ryubidragonfire @billhsia , please refer to this http://machinelearningmastery.com/multi-class-classification-tutorial-keras-deep-learning-library/
# convert integers to dummy variables (i.e. one hot encoded) dummy_y = np_utils.to_categorical(encoded_Y)

Yes, Y is the target. But (None, 10) means please prepare several target samples and all the sample are size 10. In this way, if your batch size is 10, you will feed in 10 samples, also you can use different batch size feeding different amount of samples for training. But!!! every sample should have size 10. Sample amount is not fixed, but the size of sample.

The reason size of sample is 10 is because you want to do classification over 10 class and keras expected for one-hot encoding.