Keras: 'NoneType' object has no attribute XXX

[shangyusu]

As title, I followed the example: cifar10_cnn.py, using a subset of cifar10, loading data without using (X_train, y_train), (X_test, y_test) = cifar10.load_data() but using numpy to parse the data to be like <type 'numpy.ndarray'> shape: (5000, 32, 32, 3).
Then I trained the network by setting data_augmentation = True, the training part of code was same as the example

# convert class vectors to binary class matrices
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)

model = Sequential()

model.add(Convolution2D(32, 3, 3, border_mode='same',
                        input_shape=X_train.shape[1:]))
model.add(Activation('relu'))
model.add(Convolution2D(32, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Convolution2D(64, 3, 3, border_mode='same'))
model.add(Activation('relu'))
model.add(Convolution2D(64, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(nb_classes))
model.add(Activation('softmax'))

# let's train the model using SGD + momentum (how original).
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy',
              optimizer=sgd,
              metrics=['accuracy'])

X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255
X_test /= 255

if not data_augmentation:
    print('Not using data augmentation.')
    model.fit(X_train, Y_train,
              batch_size=batch_size,
              nb_epoch=nb_epoch,
              validation_data=(X_test, Y_test),
              shuffle=True)
else:
    print('Using real-time data augmentation.')

    # this will do preprocessing and realtime data augmentation
    datagen = ImageDataGenerator(
        featurewise_center=False,  # set input mean to 0 over the dataset
        samplewise_center=False,  # set each sample mean to 0
        featurewise_std_normalization=False,  # divide inputs by std of the dataset
        samplewise_std_normalization=False,  # divide each input by its std
        zca_whitening=False,  # apply ZCA whitening
        rotation_range=0,  # randomly rotate images in the range (degrees, 0 to 180)
        width_shift_range=0.1,  # randomly shift images horizontally (fraction of total width)
        height_shift_range=0.1,  # randomly shift images vertically (fraction of total height)
        horizontal_flip=True,  # randomly flip images
        vertical_flip=False)  # randomly flip images

    # compute quantities required for featurewise normalization
    # (std, mean, and principal components if ZCA whitening is applied)
    datagen.fit(X_train)

    # fit the model on the batches generated by datagen.flow()
    model.fit_generator(datagen.flow(X_train, Y_train,
                        batch_size=batch_size),
                        samples_per_epoch=X_train.shape[0],
                        nb_epoch=nb_epoch,
                        validation_data=(X_test, Y_test))

but it threw the error:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 435, in data_generator_task
    generator_output = next(generator)
  File "/usr/local/lib/python2.7/dist-packages/keras/preprocessing/image.py", line 496, in next
    x = self.image_data_generator.random_transform(x.astype('float32'))
  File "/usr/local/lib/python2.7/dist-packages/keras/preprocessing/image.py", line 362, in random_transform
    fill_mode=self.fill_mode, cval=self.cval)
  File "/usr/local/lib/python2.7/dist-packages/keras/preprocessing/image.py", line 108, in apply_transform
    final_offset, order=0, mode=fill_mode, cval=cval) for x_channel in x]
AttributeError: 'NoneType' object has no attribute 'interpolation'

and the error was different sometimes:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 433, in data_generator_task
    generator_output = next(generator)
  File "/usr/local/lib/python2.7/dist-packages/keras/preprocessing/image.py", line 496, in next
    x = self.image_data_generator.random_transform(x.astype('float32'))
  File "/usr/local/lib/python2.7/dist-packages/keras/preprocessing/image.py", line 360, in random_transform
    transform_matrix = transform_matrix_offset_center(transform_matrix, h, w)
TypeError: 'NoneType' object is not callable

Any help would be very nice to me, thanks.

[shangyusu]

all errors were about 'NoneType' object, and if I save the weights after model.fit_generator, then there would be no exception.

[miteeek]

same problem here. Net is learning, but at the end throws an error:

Exception in thread Thread-201:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 433, in data_generator_task
    generator_output = next(generator)
  File "/usr/local/lib/python2.7/dist-packages/keras/preprocessing/image.py", line 605, in next
    x = self.image_data_generator.random_transform(x)
  File "/usr/local/lib/python2.7/dist-packages/keras/preprocessing/image.py", line 362, in random_transform
    fill_mode=self.fill_mode, cval=self.cval)
  File "/usr/local/lib/python2.7/dist-packages/keras/preprocessing/image.py", line 108, in apply_transform
    final_offset, order=0, mode=fill_mode, cval=cval) for x_channel in x]
AttributeError: 'NoneType' object has no attribute 'interpolation'

[pengpaiSH]

Exactly !!!

[burgalon]

i'm also experiencing something similar. When running the same script in the CLI, the exception disappears

[burgalon]

It's possible that I was getting this error because I was calling model.compile() without a metrics=['accuracy'] parameter

[klintan]

I'm getting the same error message as well, everytime it seems it is failing. This error is described before in the google group as well (https://groups.google.com/forum/#!topic/keras-users/C55IGYZ8hNk) , however at that time the solution was to upgrade numpy to higher than 1.10.0, I have the latest numpy 1.12 and I still get this everytime (installed using PIP).

Not sure what the cause is, anyone has any other ideas?
The model is saved, but one of the threads still throw an error. I looked into the code for image generator but couldn't find any issues.

from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
# from keras import backend as K
# K.set_image_dim_ordering('th')

# dimensions of our images.
img_width, img_height = 150, 150

train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 2000
nb_validation_samples = 800
nb_epoch = 1

model = Sequential()
model.add(Convolution2D(32, 3, 3, input_shape=(3, 150, 150)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Convolution2D(32, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Convolution2D(64, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
        rescale=1./255)


# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1./255)

# this is a generator that will read pictures found in
# subfolers of 'data/train', and indefinitely generate
# batches of augmented image data
train_generator = train_datagen.flow_from_directory(
        'data/train',  # this is the target directory
        target_size=(150, 150),  # all images will be resized to 150x150
        batch_size=32,
        class_mode='binary')  # since we use binary_crossentropy loss, we need binary labels

# this is a similar generator, for validation data
validation_generator = test_datagen.flow_from_directory(
        'data/validation',
        target_size=(150, 150),
        batch_size=32,
        class_mode='binary')

model.fit_generator(
        train_generator,
        samples_per_epoch=1024,
        nb_epoch=nb_epoch,
        validation_data=validation_generator,
        nb_val_samples=512)
model.save_weights('first_try.h5')  # always save your weights after training or during training

This code is pretty much copy paste from one of the Keras blog tutorials. But I also experience the same problems on other code using the image data generator.
I've tested it both local and on my deep learning rig. So if there was an issue with the numpy/python/keras installation should not be on all of them. Furter I installed and uninstalled both numpy and keras several times in different order.

[drgarg]

I have exactly the same problem, but I noticed something unique. I am hoping this post will help somebody figure out a solution.

So like many people here, I am trying to learn from the keras blog about learning from very little data:
https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

The blog links to 3 different codes.
code#1: https://gist.github.com/fchollet/0830affa1f7f19fd47b06d4cf89ed44d
code#2: https://gist.github.com/fchollet/f35fbc80e066a49d65f1688a7e99f069
code#3: https://gist.github.com/fchollet/7eb39b44eb9e16e59632d25fb3119975

I am able to run code #2 and #3 without problems. Only #1 gives me error. Admittedly, I am a noob when it comes to coding. Thus, my debugging skills are below average.

Things I tried without any benefit:
-adding the extra import statements from code #2 and #3
-removing save weights line at the bottom.

Unique thing:
Code #1 is very similar to Code #3, except code#1 does not use any pre-trained .h5 file.
All the image_generators are initialized and used in the same manner in code #1 and code#3

another user commented with little more insight: https://gist.github.com/fchollet/0830affa1f7f19fd47b06d4cf89ed44d#gistcomment-1904132

[lukeyeager]

I'm seeing the same. Perhaps fit_generator() sets numpy = None somewhere?

[lukeyeager]

Setting this line to

thread.daemon = False

solves the problem for me. See https://github.com/fchollet/keras/pull/4993.

[klintan]

Thanks @lukeyeager , makes sense that it is the multithreading that messes things up, especially since some people got different error messages each time.
I will try it and see if it solves things for me as well :)

[drgarg]

I will definitely try that.

Also, I found that in any generator, passing this argument reduces the frequency of the error: pickle_safe=True

EXAMPLE:
model.fit_generator(
train_generator,
samples_per_epoch=nb_train_samples,
nb_epoch=nb_epoch,
validation_data=validation_generator,
nb_val_samples=nb_validation_samples,
callbacks=[history, early_stopping],
max_q_size=4,
pickle_safe=True,
nb_worker=1)

[patyork]

pickle_safe utilizes multiprocessing, not multithreading.

I've tried to reproduce the error with python 2.7 and have not been able to.

[netw0rkf10w]

@lukeyeager I'm having the same problem in Keras 2.1.2. Unfortunately the code of this version is different from 1.2.0 and I couldn't find thread.daemon as you suggested. Do you have a solution for 2.1.2 please? Thanks in advance.

[kheffah]

Same here. I'm facing this in Keras 2.1.2 and there's no thread.daemon

[likejazz]

Same here during model.save().
Keras 2.1.4 backed by TensorFlow 1.5.0 and used model = multi_gpu_model(model)

Traceback (most recent call last):
  File "train.py", line 92, in <module>
    model.save('./data/malstm.h5')
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 2576, in save
    save_model(self, filepath, overwrite, include_optimizer)
  File "/usr/local/lib/python3.5/dist-packages/keras/models.py", line 111, in save_model
    'config': model.get_config()
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py", line 2417, in get_config
    return copy.deepcopy(config)
  File "/usr/lib/python3.5/copy.py", line 155, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.5/copy.py", line 243, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.5/copy.py", line 155, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.5/copy.py", line 218, in _deepcopy_list
    y.append(deepcopy(a, memo))
  File "/usr/lib/python3.5/copy.py", line 155, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.5/copy.py", line 243, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.5/copy.py", line 155, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.5/copy.py", line 243, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.5/copy.py", line 155, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.5/copy.py", line 223, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/usr/lib/python3.5/copy.py", line 223, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/usr/lib/python3.5/copy.py", line 155, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.5/copy.py", line 223, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/usr/lib/python3.5/copy.py", line 223, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/usr/lib/python3.5/copy.py", line 182, in deepcopy
    y = _reconstruct(x, rv, 1, memo)
  File "/usr/lib/python3.5/copy.py", line 297, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib/python3.5/copy.py", line 155, in deepcopy
    y = copier(x, memo)
  File "/usr/lib/python3.5/copy.py", line 243, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib/python3.5/copy.py", line 182, in deepcopy
    y = _reconstruct(x, rv, 1, memo)
  File "/usr/lib/python3.5/copy.py", line 306, in _reconstruct
    y.__dict__.update(state)
AttributeError: 'NoneType' object has no attribute 'update'

[walkingcloud]

NoneType means you have an empty data,mostly the labels

[creotiv]

I had problems with saving model, when used TF Hub model as head, just use save_weights, it work.

[asking28]

I had experienced the same problem. My mistake was I was trying to return object from model.compile. But I found out model.compile(...) does not return anything. Correct

model.compile(optimizer=Adam(0.0001),loss=binary_crossentropy,metrics=['acc'])