Keras: Loading a trained model, popping the last two layers, and then saving it

[robbiebarrat]

So I'm working with this architecture for a facial point network:

n_kpts = 68 # number of keypoints

input_shape = (1,100,100)

input_1 = Input(shape=input_shape)

conv_1 = Conv2D(34, kernel_size=(9,9),
                activation='tanh',
                input_shape=input_shape,
                padding='same',
                data_format='channels_first')(input_1)

conv_2 = Conv2D(34, kernel_size=(9,9),
                activation='tanh',
                padding='same',
                data_format='channels_first')(conv_1)

conv_3 = Conv2D(34, kernel_size=(9,9),
                activation='tanh',
                padding='same',
                data_format='channels_first')(conv_2)

conv_4 = Conv2D(34, kernel_size=(9,9),
                activation='tanh',
                padding='same',
                data_format='channels_first')(conv_3)

conv_5 = Conv2D(34, kernel_size=(9,9),
                activation='tanh',
                padding='same',
                data_format='channels_first')(conv_4)

softargmax = spatial_softArgmax(68)(conv_4)

reshape = Reshape((68,2))(softargmax)

model = Model(inputs=input_1, outputs=reshape)

I need to get rid of the reshape and softargmax (it's a custom layer) - and just save the model as the input and conv_1 - conv_5; I want the output to just be the output of that last convolutional layer. I have a model that's trained as an h5 with all of these layers, but i run into some trouble when trying to pop and resave - here's the script I wrote for that:

def get_weights_without_softargmax(fname):
    model = load_model(fname, custom_objects={'spatial_softArgmax':spatial_softArgmax})

    model.summary()
    model.layers.pop() # reshape layer
    model.layers.pop() # spatial softargmax

    model.summary()

    model.save("no_softargmax_" + str(fname))

The first model.summary returns a summary of the whole network;

Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 1, 100, 100)       0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 34, 100, 100)      2788      
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 34, 100, 100)      93670     
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 34, 100, 100)      93670     
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 34, 100, 100)      93670     
_________________________________________________________________
spatial_soft_argmax_1 (spati (None, 68)                0         
_________________________________________________________________
reshape_1 (Reshape)          (None, 34, 2)             0         
=================================================================
Total params: 283,798
Trainable params: 283,798
Non-trainable params: 0

and the second one returns the now popped network

Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         (None, 1, 100, 100)       0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 34, 100, 100)      2788      
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 34, 100, 100)      93670     
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 34, 100, 100)      93670     
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 34, 100, 100)      93670     
=================================================================
Total params: 283,798
Trainable params: 283,798
Non-trainable params: 0

but when I try to do model.save() - i get this error:

Traceback (most recent call last):
  File "weight_chopper.py", line 18, in <module>
    get_weights_without_softargmax("34pts_94percent.h5")
  File "weight_chopper.py", line 16, in get_weights_without_softargmax
    model.save("no_softargmax_" + str(fname))
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 2553, in save
    save_model(self, filepath, overwrite, include_optimizer)
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 107, in save_model
    'config': model.get_config()
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 2390, in get_config
    new_node_index = node_conversion_map[node_key]
KeyError: u'reshape_1_ib-0'

Note how it's referencing the old reshape layer? When I defined the model; I said model = Model(inputs=input_1, outputs=reshape) - so does it still think that the model has that reshape output? How can I convince it otherwise? I've tried doing another Model(inputs=..., outputs=...) type command; but there aren't any appropriate values to plug in for the inputs and outputs!

How can I get the model to save (preferably as a compiled model) with just the convolutional layers?

[fchollet]

Build your model before calling save: model.build(). I think that should solve it.

[robbiebarrat]

@fchollet thanks so much for the quick reply; what arguments do i feed model.build(), though?
It's saying TypeError: build() takes exactly 2 arguments (1 given) - and I couldn't find anything on this after a scan of the documentation.

[fchollet]

model.build(None). It normally takes an input shape, but in the case of a Sequential model that's not relevant.

[fchollet]

Wait, I got confused. I thought you were using the pop method of a Sequential model, but that's not what you are doing. Please post your full code.

Note that pop is not possible with the functional API, it's only implemented for Sequential. If you want to drop some layers in the functional API, you'd do:

new_model = Model(inputs=input_1, outputs=conv_5)

In you case.

[mturnshek]

Try something like this:

import keras
from keras.models import Model, load_model
from keras.layers import Input, Dense
from keras.optimizers import RMSprop
import numpy as np


# Create original model and save it
inputs = Input((1,))
dense_1 = Dense(10, activation='relu')(inputs)
dense_2 = Dense(10, activation='relu')(dense_1)
dense_3 = Dense(10, activation='relu')(dense_2)
outputs = Dense(10)(dense_3)

model = Model(inputs=inputs, outputs=outputs)
model.compile(optimizer=RMSprop(), loss='mse')
model.save('test.h5')


# Load the model and make modifications to it
loaded_model = load_model('test.h5')
loaded_model.layers.pop()
loaded_model.layers.pop()


# Create your new model with the two layers removed and transfer weights
new_model = Model(inputs=inputs, outputs=dense_1)
new_model.compile(optimizer=RMSprop(), loss='mse')
new_model.set_weights(loaded_model.get_weights())

new_model.summary()
new_model.save('test_complete.h5')

[robbiebarrat]

@fchollet I guess I was mistakenly using the pop method; wasn't aware that it only worked on Sequential models - in this case, I can't do the new_model = Model(inputs=input_1, outputs=conv_5) - because I have already trained and saved the weights of the full model - When loading that model; there aren't any values i can give for inputs= and outputs= because I haven't defined the model in that script...

As for my full code; here is where i define, train, and save my model: https://gist.github.com/robbiebarrat/5bbd42c9f485198b0f37f6e36ca81a14

here is the attempted "layer removal" script where I load that model, remove the last two layers, and save it again:

https://gist.github.com/robbiebarrat/161d88ff6e2548d87d98a70129e56b95

[fchollet]

there aren't any values i can give for inputs= and outputs= because I haven't defined the model in that script...

You can retrieve these from your model:

new_model = Model(model.inputs, model.layers[-3].output)  # assuming you want the 3rd layer from the last

[robbiebarrat]

@fchollet oh sick - i think that worked; got no errors and the model summary looks good.

Here's the final code I used:

def get_weights_without_softargmax(fname):
    model = load_model(fname, custom_objects={'spatial_softArgmax':spatial_softArgmax})

    model.summary()

    new_model = Model(model.inputs, model.layers[-3].output)

    new_model.summary()

    new_model.set_weights(model.get_weights())

    new_model.save("no_softargmax_" + str(fname))

get_weights_without_softargmax("34pts_94percent.h5")

[robbiebarrat]

Only concern is that the new model is 1.2 MB, and the old one was 2.3 MB - the file size shouldn't have decreased so much; as the last two layers didn't have any trainable parameters?

It'll take me a little bit to test it and finally figure it out; as I need to write the program that interprets the filters output by the network. Feel free to close the issue for right now while I'm figuring that out, and if all goes well I won't reopen it.

Thanks so much!

[Shawn-Shan]

You can use the following function to cut a model and obtain a new model with the original input and given layer output.
def build_bottleneck_model(model, layer_name): for layer in model.layers: if layer.name == layer_name: output = layer.output bottleneck_model = Model(model.input, output) bottleneck_model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) return bottleneck_model

[myurasov]

Per @fchollet recipe, but using recommended get_input_at/get_output_at, loading pre-trained VGG19, and stripping predictions layer:

# create model
vgg = keras.applications.vgg19.VGG19(include_top=True, weights='imagenet')

# remove predictions layer
vgg = keras.models.Model(inputs=vgg.get_input_at(0), outputs=vgg.layers[-2].get_output_at(0))

[wt-huang]

Closing as this is resolved

[keineahnung2345]

What should I do if I want to add layers after setting weights?

new_model = Model(model.inputs, model.layers[-4].output)
new_model.set_weights(model.get_weights())
new_model.add(Conv2D(12, (1, 1))) #AttributeError: 'Model' object has no attribute 'add'
...

Is there any workaround?

EDIT:
Workaround found: https://stackoverflow.com/questions/45755022/cannot-add-layers-to-saved-keras-model-model-object-has-no-attribute-add.

One should use functional API in this situation:

conv = Conv2D(12, (1, 1))(model.layers[-4].output)
...
final_layer = ...
new_model = Model(model.inputs, final_layer)

[nda97531]

@fchollet oh sick - i think that worked; got no errors and the model summary looks good.

Here's the final code I used:

def get_weights_without_softargmax(fname):
  model = load_model(fname, custom_objects={'spatial_softArgmax':spatial_softArgmax})

  model.summary()

  new_model = Model(model.inputs, model.layers[-3].output)

  new_model.summary()

  new_model.set_weights(model.get_weights())

  new_model.save("no_softargmax_" + str(fname))

get_weights_without_softargmax("34pts_94percent.h5")

Actually you don't need to call set_weights() on new model, its weights are copied from your old one.

[kolingv]

You can use the following function to cut a model and obtain a new model with the original input and given layer output.

def build_bottleneck_model(model, layer_name):
    for layer in model.layers:
        if layer.name == layer_name:
            output = layer.output
    bottleneck_model = Model(model.input, output)
    bottleneck_model.compile(loss='categorical_crossentropy',
                             optimizer='adam',
                             metrics=['accuracy'])
    return bottleneck_model```

What if i wanna cut the model as two parts and then obtian the first part as model1 and the second part as model2? I'm doing this because tflite conversion cannot support Lambda Layer or customized layer.