Keras: How to get encoded images in the keras autoencoder blog?

[mojovski]

Please make sure that the boxes below are checked before you submit your issue. If your issue is an implementation question, please ask your question on StackOverflow or join the Keras Slack channel and ask there instead of filing a GitHub issue.

Thank you!

• [x ] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

• [ ] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

• [x ] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

• [ x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

When I use the autoencoder example from the keras blog
https://blog.keras.io/building-autoencoders-in-keras.html
I would like to generate encoded data samples (encoded images).
This works well in the simple first example but the example lacks of at least one line on how to build an encoder.

Here is the gist:
https://gist.github.com/mojovski/8509ccad123184020cd32999f94705b4

And here is the issue in a nut shell:

```
input_img = Input(shape=(1, 28, 28))

x = Convolution2D(16, 3, 3, activation='relu', border_mode='same')(input_img)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
encoded = MaxPooling2D((2, 2), border_mode='same')(x)

"""

at this point the representation is (8, 4, 4) i.e. 128-dimensional

"""
encoded_input = Input(shape=(8,4,4)) #<--------- ? How to get this shape from the encoded layer?

x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(16, 3, 3, activation='relu')(x)
x = UpSampling2D((2, 2))(x)
decoded = Convolution2D(1, 3, 3, activation='sigmoid', border_mode='same')(x)

autoencoder = Model(input_img, decoded)
encoder=Model(input_img, encoded)

decoder=Model(encoded_input, decoded) <---------------- THIS FAILS

"""
error if enabled:
Graph disconnected: cannot obtain value for tensor input_1 at layer "input_1". The following previous layers were accessed without issue: []
"""

autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

Thank you very much!

[unrealwill]

Hello,
encoded_input = Input(shape=(8,4,4)) #<--------- ? How to get this shape from the encoded layer?
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(encoded)
should have been
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(encoded_input)

[mojovski]

Thank you very much!

but how can I then train the whole autoencoder and then split the modules? Later, I would like to use only the encoder and the decoder as stand alone models.

[unrealwill]

Just keep a reference to decoder and encoder, for later use. As they shared the weights with autoencoder, once you have trained the autoencoder, the weights of the decoder model and encoder model have been modified.

[renxinhe]

I ran into this same issue. I wonder if you were able to extract the value of encoder.

What I did is that I saved a reference to encoder after the whole model is fitted. I then tried to run the encoder is a TF session in an attempt to get the encoded images.

sess = tf.Session()
predicted_states = sess.run([encoder])

Of couse, encoder needs a feed dict to give it the input image. I got the following, expected, error message:

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input_1' with dtype float
[[Node: input_1 = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

I'm now stuck on how to actually get the reference to the placeholder variable that Keras silently created. Normally in TensorFlow, we do x = tf.placeholder(), so we can later feed x some value. But in this example, Keras doesn't really expose the placeholder variable. Is there a way to go around that?

[renxinhe]

I found a way to get the output of the encoder by using the Keras model API shown in the FAQ.

Here's my functional code snippet:

INPUT_SHAPE = (100, 248, 3)
input_img = Input(shape=INPUT_SHAPE)  # adapt this if using `channels_first` image data format

x = Conv2D(16, (3, 3), activation='relu', padding='same')(input_img)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
encoder = MaxPooling2D((2, 2), padding='same', name='encoder')(x)

# at this point the representation is (4, 4, 8) i.e. 128-dimensional

x = Conv2D(8, (3, 3), activation='relu', padding='same')(encoder)
x = UpSampling2D((2, 2))(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation='relu')(x)
x = UpSampling2D((2, 2))(x)
decoder = Conv2D(3, (3, 3), activation='sigmoid', padding='same')(x)

autoencoder = Model(input_img, decoder)
autoencoder.compile(loss='mse', optimizer='adam')
self.autoencoder = autoencoder

self.encoder = Model(inputs=self.autoencoder.input,
                        outputs=self.autoencoder.get_layer('encoder').output)

The last line creates a new encoder model, and I can execute the following elsewhere to get the encoded value:

encoded_states = self.encoder.predict(input)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[sanakhamekhem]

If the representation is (n, m, p) of the encoder, So, how can I reshape it to a matrix in order to fed it to another classifier.
output=encoded_imgs[1].reshape(4, 4 * 8).T ## at this point the representation is (4, 4, 8) i.e. 128-dimensional
How can I generalize it??

[jmren168]

The same question as @sanakhamekhem : how to get the compressed representation of encoder?

[sanakhamekhem]

features = model.predict(x)
feat=features.reshape(1, -1).T

[amirieb]

Here is a good solution: instead of

autoencoder = Model(input_img, decoder)

use

autoencoder = Model(inputs, decoder(encoder(inputs)))

see: this link