Keras: Loading weights for part of the network

Created on 2 Mar 2016 · 12Comments · Source: keras-team/keras

Hello,

Say I have weights from a trained network and now I want to change the architecture of the network by adding 2 more filters for the last convolutional layer. For the new network, I don't want to train it from scratch again, I want to load the existing weights to the network while initializing the weights for the added 2 filters using other techniques. Is there a way to do it in Keras?

Source

KlaymenGC

👍5

Most helpful comment

For a Sequential model, you can get the layers by index:

weights = model.layers[5].get_weights()

Or by name, if all your layers are named;

layer_dict = dict([(layer.name, layer) for layer in model.layers])
weights = layer_dict['some_name'].get_weights()

fchollet on 2 Mar 2016

👍32

All 12 comments

You could take a look here and in #1728.

tboquet on 2 Mar 2016

@tboquet Thanks for your reply, I think I don't have trouble adding weights to layers. The thing is, I'm now using a sequential model, say the last convolutional layer is:
Convolution2D(10, 3, 3, activation='relu')
and I've got the trained weights, then I change the last conv layer to:
Convolution2D(12, 3, 3, activation='relu')
and I still want to use the trained weights for the first 10 filters while initializing the weights of the remaining filters using he_normal or something like that.
Do you think it's possible? Could you please elaborate?

KlaymenGC on 2 Mar 2016

According to PR #1796, the canonical way to do this is to build both the old and the new model (you don't need to compile the old one), load weights to the old one, then use the .nodes property to manually transfer the weights to the new model.

pasky on 2 Mar 2016

@KlaymenGC oops sorry I didn't understood your question, @pasky is right!

tboquet on 2 Mar 2016

@pasky thanks for the reply, but it seems that sequential models don't have this node property... any ideas?

KlaymenGC on 2 Mar 2016

I'd try .layers for Sequential models - sorry, it's been long I've been using these.

pasky on 2 Mar 2016

For a Sequential model, you can get the layers by index:

weights = model.layers[5].get_weights()

Or by name, if all your layers are named;

layer_dict = dict([(layer.name, layer) for layer in model.layers])
weights = layer_dict['some_name'].get_weights()

fchollet on 2 Mar 2016

👍32

@fchollet yeah but adding weights from
Convolution2D(10, 3, 3, activation='relu')
to
Convolution2D(12, 3, 3, activation='relu') while initializing the weights for the last 2 filters using built-in initialization methods is not possible I guess

KlaymenGC on 2 Mar 2016

Sure it's possible. But it will require manipulating the weight array in
Numpy-space before calling set_weights(). e.g. array[:-2, :, :, :] =
old_array[:, ;, :, :]

On 2 March 2016 at 14:03, GC [email protected] wrote:

@fchollet https://github.com/fchollet yeah but adding weights from
Convolution2D(10, 3, 3, activation='relu')
to
Convolution2D(12, 3, 3, activation='relu') while initializing the weights
for the last 2 filters using built-in initialization methods is not
possible I guess

—
Reply to this email directly or view it on GitHub
https://github.com/fchollet/keras/issues/1873#issuecomment-191453685.