Keras: Dimension problem when building convolutional autoencoder

[pabloxrl]

Hi all,

I'm taking my first steps in Keras and struggling with the dimensions of my layers. I'm currently building a convolutional autoencoder that I would like to train using the MNIST dataset. Unfortunately, I cannot seem to get the dimensions right, and I'm having trouble to understand where is my mistake.

My model is built through:
`
def build_model(nb_filters=32, nb_pool=2, nb_conv=3):
input_img = Input(shape=(1, 28, 28))

x = Convolution2D(16, 3, 3, activation='relu', border_mode='same')(input_img)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
encoded = MaxPooling2D((2, 2), border_mode='same')(x)

x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(16, 3, 3, activation='relu')(x)
x = UpSampling2D((2, 2))(x)
decoded = Convolution2D(1, 3, 3, activation='sigmoid', border_mode='same')(x)

return Model(input_img, decoded)

`

and the data is retrieved using:

`
def load_data():
(x_train, _), (x_test, _) = mnist.load_data()

x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_train = np.reshape(x_train, (len(x_train), 1, 28, 28))
x_test = np.reshape(x_test, (len(x_test), 1, 28, 28))
return x_train, x_test

`

As you see, I'm trying to normalize the images to display them in black and white, and simply to train an autoencoder to be able to restore them.

Below you can see the error I'm getting:

`
Using Theano backend.

Layer (type) Output Shape Param # Connected to

input_1 (InputLayer) (None, 1, 28, 28) 0

convolution2d_1 (Convolution2D) (None, 1, 28, 16) 4048 input_1[0][0]

maxpooling2d_1 (MaxPooling2D) (None, 1, 14, 16) 0 convolution2d_1[0][0]

convolution2d_2 (Convolution2D) (None, 1, 14, 8) 1160 maxpooling2d_1[0][0]

maxpooling2d_2 (MaxPooling2D) (None, 1, 7, 8) 0 convolution2d_2[0][0]

convolution2d_3 (Convolution2D) (None, 1, 7, 8) 584 maxpooling2d_2[0][0]

maxpooling2d_3 (MaxPooling2D) (None, 1, 4, 8) 0 convolution2d_3[0][0]

convolution2d_4 (Convolution2D) (None, 1, 4, 8) 584 maxpooling2d_3[0][0]

upsampling2d_1 (UpSampling2D) (None, 2, 8, 8) 0 convolution2d_4[0][0]

convolution2d_5 (Convolution2D) (None, 2, 8, 8) 584 upsampling2d_1[0][0]

upsampling2d_2 (UpSampling2D) (None, 4, 16, 8) 0 convolution2d_5[0][0]

convolution2d_6 (Convolution2D) (None, 2, 14, 16) 1168 upsampling2d_2[0][0]

upsampling2d_3 (UpSampling2D) (None, 4, 28, 16) 0 convolution2d_6[0][0]

convolution2d_7 (Convolution2D) (None, 4, 28, 1) 145 upsampling2d_3[0][0]

Total params: 8273

Traceback (most recent call last):
File "C:/Users//Documents/GitHub/main/research/research_framework/experiment.py", line 48, in
callbacks=[EarlyStopping(patience=3)])
File "C:\Users\AppData\Local\Continuum\Anaconda2\lib\site-packages\keras\engine\training.py", line 1047, in fit
batch_size=batch_size)
File "C:\Users\AppData\Local\Continuum\Anaconda2\lib\site-packages\keras\engine\training.py", line 978, in _standardize_user_data
exception_prefix='model target')
File "C:\Users\AppData\Local\Continuum\Anaconda2\lib\site-packages\keras\engine\training.py", line 111, in standardize_input_data
str(array.shape))
Exception: Error when checking model target: expected convolution2d_7 to have shape (None, 4, 28, 1) but got array with shape (60000L, 1L, 28L, 28L)

Process finished with exit code 1
`

Could you help me to decyper this error? Are there any materials beyond Keras website about building models and dealing with this kind of issues?

Cheers

Please make sure that the boxes below are checked before you submit your issue. Thank you!

• [x] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps
• [x] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps
• [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

[kgrm]

There appears to be a conflict in your keras global settings between the backend and image dimension ordering. Look at your ~/.keras/keras.json file. If you're using the tensorflow backend, image_dim_ordering should be tf, and if you're using theano, it should be th.

[pabloxrl]

Thank you kgrm. I was going to write the same thing. I changed my json file to be:

{
    "image_dim_ordering": "th", 
    "epsilon": 1e-07, 
    "floatx": "float32", 
    "backend": "theano"
}

And it did the trick

[JoshuaZe]

i have the same problem in tensorflow backend when running the same code, but i did the same change on my json file (using "image_dim_ordering": "th" with "backend": "tensorflow"). so weird.

[matthias-samwald]

I am experiencing this error as well, even though I am using tensorflow and have 'image_dim_ordering set to 'tf' as suggested

[kgrm]

@matthias-samwald Post your code and the output including error messages.

[matthias-samwald]

It's basically the same tutorial code of pabloxrl, just that I'm using Tensorflow backend.

Python version: 3.5
Tensorflow version: 0.11.0rc2
Keras version: 1.1.1

Code:
https://gist.github.com/matthias-samwald/b255197c446f7ee2d7a46faeb1fdbc75

Error (triggered by calling autoencoder.fit)

 tensorflow/core/common_runtime/gpu/gpu_device.cc:1041] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
Traceback (most recent call last):
  File "/home/matthias/Documents/Intelligence/keras-examples/AE_tutorial_3_convolutional.py", line 42, in <module>
    callbacks=[TensorBoard(log_dir='/tmp/autoencoder_tensorboard')])
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 1052, in fit
    batch_size=batch_size)
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 983, in _standardize_user_data
    exception_prefix='model target')
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/training.py", line 111, in standardize_input_data
    str(array.shape))
Exception: Error when checking model target: expected convolution2d_7 to have shape (None, 4, 28, 1) but got array with shape (60000, 1, 28, 28)

Process finished with exit code 1

keras.json:
{ "floatx": "float32", "backend": "tensorflow", "image_dim_ordering": "tf", "epsilon": 1e-07 }

[kgrm]

You're reshaping the MNIST data to the Theano dimension conventions. TF shape convention for image tensors is (N_samples, height, width, N_channels).

[matthias-samwald]

I see. The tutorial did not mention that the code was specific to the Theano backend. Thanks!

[Iragne]

You can change this to make it work on tf

x_train = np.reshape(x_train, (len(x_train),  28, 28, 1))
x_test = np.reshape(x_test, (len(x_test), 28, 28, 1))

input_img = Input(shape=(28, 28,1))

[cclough]

Thanks @iragne - I had the same problem and that fixed it

[metatl]

I tried all of the above and the same issue exists.

I'm using keras 1.1.1 with theano as a backend

keras json file:
{"epsilon": 1e-07, "floatx": "float32", "backend": "theano","image_dim_ordering":"th"}

Running the sample code is fine. I modified the code a little bit to suit my purpose:

from keras.layers import Dense, Convolution2D, MaxPooling2D, UpSampling2D, Input
from keras.models import Model
import numpy as np

input_img = Input(shape=(1, 73, 144))

x = Convolution2D(16, 3, 3, activation='relu', border_mode='same')(input_img)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(12, 3, 3, activation='relu', border_mode='same')(x)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(6, 3, 3, activation='relu', border_mode='same')(x)
encoded = MaxPooling2D((2, 2), border_mode='same')(x)

at this point the representation is (8, 4, 4) i.e. 128-dimensional

x = Convolution2D(6, 3, 3, activation='relu', border_mode='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(12, 3, 3, activation='relu', border_mode='same')(x)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(16, 3, 3, activation='relu')(x)
x = UpSampling2D((2, 2))(x)
decoded = Convolution2D(1, 3, 3, activation='sigmoid', border_mode='same')(x)
print decoded.shape
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

data = np.load('data.npy')

random_select = np.random.choice(456,45)
data_test = sst[random_select,:,:]
data_train = np.delete(sst,random_select,axis = 0)

data_test = data_test/data_test.max()
data_train = data_train/data_train.max()

data_test = np.reshape(data_test,[len(random_select),1,73,144])
data_train = np.reshape(data_train,[456-len(np.unique(random_select)),1,73,144])
print data_test.shape, data_train.shape
autoencoder.fit(data_train, data_train,nb_epoch= 100 ,batch_size=10, shuffle=True, \
validation_data=(data_test, data_test))

So basically, I just changed the input img size. I got:

(45, 1, 73, 144) (413, 1, 73, 144)
Traceback (most recent call last):
File "conv_encoder.py", line 41, in
validation_data=(sst_test, sst_test))
File "/usr/local/other/SSSO_Ana-PyD/2.4.0_py2.7/lib/python2.7/site-packages/keras/engine/training.py", line 1052, in fit
batch_size=batch_size)
File "/usr/local/other/SSSO_Ana-PyD/2.4.0_py2.7/lib/python2.7/site-packages/keras/engine/training.py", line 983, in _standardize_user_data
exception_prefix='model target')
File "/usr/local/other/SSSO_Ana-PyD/2.4.0_py2.7/lib/python2.7/site-packages/keras/engine/training.py", line 111, in standardize_input_data
str(array.shape))
Exception: Error when checking model target: expected convolution2d_7 to have shape (None, 1, 76, 140) but got array with shape (413, 1, 73, 144)

Any idea, what is wrong here?

[metatl]

Issue solved. My own mistake.

1. I should have added a "border_mode='same'" to one of the convo layer.
2. My input image size is 73, which screwed up the dimension bad.

[Ethiral]

Newbie here.Can someone please tell me how to view and edit json file from command line.Thank You

[steffipriyanka]

Hi am new to keras . I also have the same dimension issue. Can anyone kindly help me with this pls.

this is my code.
CIFAR_CNN.txt
am using keras 2.0.5
tensorflow backend
tensorflow 1.1.0
theano 0.10