Keras: Fine-tuning pre-trained VGG16 not possible since `add` method is not defined for `Model` class?

[jrosebr1]

I am trying to fine-tune the pre-trained VGG16 network from keras.applications.VGG16. I'm doing the standard approach that @fchollet detailed in his blog post.

My code is as follows:

# load the VGG16 network, ensuring the head FC layer sets are
# left off
print("[INFO] loading VGG16...")
model = VGG16(weights="imagenet", include_top=False)

# loop over the layers in VGG (until the final CONV block) and
# freeze the layers -- we will only be fine-tuning the final CONV
# block along with our dense FC layers
for layer in model.layers[:15]:
    layer.trainable = False

# load the FCHeadNet and add it to the convolutional base
print("[INFO] loading head...")
head = FCHeadNet.build((512 * 7 * 7,), 17, dropout=True)
head.load_weights(args["head"])
model.add(head)

The FCHeadNet class simply defines a Sequential model. However, when I try to add head to the model I receive the following error message:

File "finetune.py", line 30, in 
    model.add(head)
AttributeError: 'Model' object has no attribute 'add'

Inspecting the vgg16.py source I see that VGG16 is defined as a Model versus a Sequential, thus there is no .add method. My question is therefore:

_How do I fine-tune the pre-trained VGG16 class? Or is this simply not possible and I need to define VGG16 by hand and load the weights manually?_

[fchollet]

You should recover the output you want to build on top of, and use it to instantiate a new model.

If you want to use an intermediate layer, you can use model.get_layer(name). If you want to use the last layer, you can just use model.output.

initial_model = VGG16(weights="imagenet", include_top=False)
last = model.output

x = Flatten()(last)
x = Dense(1024, activation='relu')(x)
preds = Dense(200, activation='softmax')(x)

model = Model(initial_model.input, preds)

This is detailed in the docs, too: https://keras.io/applications/#fine-tune-inceptionv3-on-a-new-set-of-classes

[parikshit95]

I am trying to get accuracy from my predicted results from the VGG16 model. However, the decode_predictions() function only returns a tuple containing ID and Label and not accuracy. Is there any way for decode_predictions() to return accuracy as well?
model = VGG16(weights="imagenet", include_top=True)
sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9,nesterov=True)
model.compile(optimizer=sgd,loss='categorical_crossentropy',metrics=['accuracy'])
print("network loaded")

Get Lable for the images in directory

preds = []
i=0
for image in data:
preds.append(model.predict(image))
i = i+1
print(i)
#print("[INFO] loss={:.4f}, accuracy: {:.4f}%.format(loss,accuracy*100)")

Decode the predictions

decode = []
for preds in preds:
decode.append(decode_predictions(preds, top=3)[0])
print(decode)

[ptisseur]

With Keras 2.02 this does not work
base_model = applications.VGG16(include_top=False, weights='imagenet')
last = base_model.output
x = Flatten()(last)
model = Model(input=base_model.input, output=x)

Trying with
base_model = applications.VGG16(include_top=False, weights='imagenet')
last = base_model.output
x=Flatten(input_shape=base_model.output_shape[1:])(x)
model = Model(input=base_model.input, output=x)
This does not work

With AveragePooling2D()(x) there is no more errors

x=Flatten(input_shape=base_model.output_shape[1:])(x)

x=Dense(64, activation='relu')(x)

model.add(Dropout(0.5)

x=Dense(8, activation='softmax')(x)

this is the model we will train

model = Model(input=base_model.input, output=x)

Anyone can help me?

[riccardosamperna]

Same problem here, it is not able to recover the size of the base_model.

[SinghGauravKumar]

@ptisseur @riccardosamperna Did you guys figure it out?

[twang9]

@ptisseur My code works. Add a prediction layer before "model=Model(...)". If still not working, try to rename the prediction layer.

[edoven]

I solved by creating "new_model = Sequential()". I then copied all the layers from "applications.VGG16(..)" into this new model.
This way I get a normal Sequential model and I can do the concatenation "new_model.add(top_model)".
Hope this helps.

from keras import applications, optimizers
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Dropout, Flatten, Dense, Input
import keras

print(keras.__version__)
# OUTPUT: '2.0.3'

keras.backend.set_image_dim_ordering('tf')


# SET ALL THE PARAMETERS
weights_path = 'models/vgg16.h5'
top_model_weights_path = 'bottleneck_fc_model.h5'
img_width, img_height = 150, 150
train_data_dir = 'data/dogscats1000/train'
validation_data_dir = 'data/dogscats1000/validation'
nb_train_samples = 2000
nb_validation_samples = 2000
epochs = 30
batch_size = 16



# LOAD VGG16
input_tensor = Input(shape=(150,150,3))
model = applications.VGG16(weights='imagenet', 
                           include_top=False,
                           input_tensor=input_tensor)


# CREATE A TOP MODEL
top_model = Sequential()
top_model.add(Flatten(input_shape=model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(1, activation='sigmoid'))
top_model.load_weights(top_model_weights_path)


# CREATE AN "REAL" MODEL FROM VGG16
# BY COPYING ALL THE LAYERS OF VGG16
new_model = Sequential()
for l in model.layers:
    new_model.add(l)


# CONCATENATE THE TWO MODELS
new_model.add(top_model)

# LOCK THE TOP CONV LAYERS
for layer in new_model.layers:
    layer.trainable = False

# COMPILE THE MODEL
new_model.compile(loss='binary_crossentropy',
              optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
              metrics=['accuracy'])



# CREATE THE IMAGE GENERATORS
train_datagen = ImageDataGenerator(rescale=1./255,
                                   shear_range=0.2,
                                   zoom_range=0.2,
                                   horizontal_flip=True)
validation_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
                        train_data_dir,
                        target_size=(img_height,img_width),
                        batch_size=batch_size,
                        class_mode='binary')

validation_generator = validation_datagen.flow_from_directory(
                            validation_data_dir,
                            target_size=(img_height,img_width),
                            batch_size=batch_size,
                            class_mode='binary')


#  FIT THE MODEL

new_model.fit_generator(
    train_generator,
    samples_per_epoch=nb_train_samples,
    epochs=epochs,
    validation_data=validation_generator,
    nb_val_samples=nb_validation_samples)

[Ksen17]

@edoven
I tried to use Your code but accuracy falls down to 0.0195

[fedden]

EDIT: I realised actually looking at the VGG model, that the tutorial might be wrong? The tutorial suggests we block all layers up to 25, but that would freeze all the layers in VGG16!

If we print the summary of VGG16 then it looks like the first 15 layers the three convolutional blocks that we want to freeze and the last block that we want to unfreeze seems to begin from layer 15.

---

@edoven Thanks for the help on creating a new model out of the VGG model!

I believe your code a little bit incorrect with respect to the original tutorial, which instructs the first 25 (but I think it should be 15) layers should be frozen, but not all of them; the last layers comprise the last convolutional block and are to be finetuned with the top model.

Forgive (and correct) me if I'm wrong, but I think your code should be changed from this:

# LOCK THE TOP CONV LAYERS
for layer in new_model.layers:
    layer.trainable = False

to this:

# LOCK THE TOP CONV LAYERS
for layer in new_model.layers[:15]:
    layer.trainable = False

[pabloskyg]

@fchollet Thank you very much for sharing your codes, a great men you. And @edoven Excellent contribution¡¡, at least it works for me. Thank you very much for you help¡¡¡. My version of code for spanish speaking people.

from keras import applications
from keras.preprocessing.image import ImageDataGenerator
from keras import optimizers
from keras.models import Sequential
from keras.layers import Dropout, Flatten, Dense, Input
import keras

print(keras.__version__)
# OUTPUT: '2.2.4'

# Ruta de los archivos con los pesos del modelo entrenado antes
weights_path = '/home/investigacion-desarrollo/Escritorio/prueba_lab/Ejemplo2_perros_gatos/vgg16_weights.h5'

top_model_weights_path = '/home/investigacion-desarrollo/Escritorio/prueba_lab/Ejemplo2_perros_gatos/fc_model.h5'

# Dimensiones de nuestras imagenes
img_width, img_height = 150, 150

train_data_dir = '/home/investigacion-desarrollo/Escritorio/prueba_lab/Ejemplo2_perros_gatos/data/train'

validation_data_dir = '/home/investigacion-desarrollo/Escritorio/prueba_lab/Ejemplo2_perros_gatos/data/validation'

nb_train_samples = 2000
nb_validation_samples = 800
epochs = 50
batch_size = 16

# Compilando una red VGG16
input_tensor = Input(shape=(150,150,3))
model = applications.VGG16(weights='imagenet', include_top=False, input_tensor=input_tensor)

# Compilando un modelo clasificador para colocar en la parte superior (TOP) de la red convolucional
top_model = Sequential()

# Aqui colocamos como entrada el resultado de la VGG16
top_model.add(Flatten(input_shape=model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))

# Aqui utilizamos 1 salida por que estamos haciendo una clasificacion binaria
top_model.add(Dense(1, activation='sigmoid'))

# Notar que esto es necesario para comenzar con un clasificador completamente entrenado, incluyendo al clasificador de capas superiores, con el fin de hacer satisfactoria la sintonia-fina.
top_model.load_weights(top_model_weights_path)

# Agregar (add) el modelo a la parte superior(TOP) de la base convolucional. Es decir, creamos un modelo "REAL" desde VGG16 mediante la copia de todas las capas de VGG16.
new_model = Sequential()
for l in model.layers:
    new_model.add(l)

# Concatenando los dos modelos
new_model.add(top_model)

# Bloqueo (o congelamiento) de las capas superiores (TOP)
for layer in new_model.layers:
    layer.trainable = False


# Compilar el modelo con un optimizador SGD/momentum y una muy baja tasa de aprendizaje.
new_model.compile(loss='binary_crossentropy',
              optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
              metrics=['accuracy'])

# Preparar la configuracion para el aumento de datos. Es decir creamos un generador de imagenes
train_datagen = ImageDataGenerator(
    rescale     = 1./255,
    shear_range = 0.2,
    zoom_range  = 0.2,
    horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale=1./ 255)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='binary')

validation_generator = test_datagen.flow_from_directory(
    validation_data_dir,
    target_size=(img_height, img_width),
    batch_size=batch_size,
    class_mode='binary')

# Sintonia fina del modelo
new_model.fit_generator(
    train_generator,
    samples_per_epoch=nb_train_samples,
    epochs=epochs,
    validation_data=validation_generator,
    nb_val_samples=nb_validation_samples)