Keras: ImageDataGenerator now longer allows preprocessing_functions from keras.applications to be used

[kmader]

It appears that after this pull: https://github.com/keras-team/keras/pull/9273
Is it now impossible to use the ImageDataGenerator with the preprocessing functions?

from keras.preprocessing.image import ImageDataGenerator
from keras.applications.xception import preprocess_input

core_idg = ImageDataGenerator(samplewise_center=False, 
                              samplewise_std_normalization=False, 
                              horizontal_flip = True, 
                              vertical_flip = False, 
                              height_shift_range = 0.15, 
                              width_shift_range = 0.15, 
                              rotation_range = 5, 
                              shear_range = 0.01,
                              fill_mode = 'nearest',
                              zoom_range=0.25,
                             preprocessing_function = preprocess_input)

next(core_idg.flow_from_directory(image_dir))

Results in

TypeError: unsupported operand type(s) for /=: 'PngImageFile' and 'float'

Wrapping preprocessing_function to it first turns the data to an array doesn't work either
ppi = lambda x: preprocess_input(np.array(x).astype(np.float32))

Since it seems to expect a PIL image (which also doesn't work since it is a float-type)

TypeError: 'tuple' object cannot be interpreted as an integer

Detailed Error Message

TypeError                                 Traceback (most recent call last)
<ipython-input-40-2755df92388f> in <module>()
     18                             target_size = IMG_SIZE,
     19                              color_mode = 'rgb',
---> 20                             batch_size = 1024)) # one big batch

/usr/local/lib/python3.6/dist-packages/keras/preprocessing/image.py in __next__(self, *args, **kwargs)
    858 
    859     def __next__(self, *args, **kwargs):
--> 860         return self.next(*args, **kwargs)
    861 
    862     def _get_batches_of_transformed_samples(self, index_array):

/usr/local/lib/python3.6/dist-packages/keras/preprocessing/image.py in next(self)
   1281         # The transformation of images is not under thread lock
   1282         # so it can be done in parallel
-> 1283         return self._get_batches_of_transformed_samples(index_array)

/usr/local/lib/python3.6/dist-packages/keras/preprocessing/image.py in _get_batches_of_transformed_samples(self, index_array)
   1231                            interpolation=self.interpolation)
   1232             if self.image_data_generator.preprocessing_function:
-> 1233                 img = self.image_data_generator.preprocessing_function(img)
   1234             if self.target_size is not None:
   1235                 width_height_tuple = (self.target_size[1], self.target_size[0])

/usr/local/lib/python3.6/dist-packages/keras/applications/xception.py in preprocess_input(x)
    279         Preprocessed array.
    280     """
--> 281     return imagenet_utils.preprocess_input(x, mode='tf')

/usr/local/lib/python3.6/dist-packages/keras/applications/imagenet_utils.py in preprocess_input(x, data_format, mode)
    176     else:
    177         return _preprocess_symbolic_input(x, data_format=data_format,
--> 178                                           mode=mode)
    179 
    180 

/usr/local/lib/python3.6/dist-packages/keras/applications/imagenet_utils.py in _preprocess_symbolic_input(x, data_format, mode)
    113 
    114     if mode == 'tf':
--> 115         x /= 127.5
    116         x -= 1.
    117         return x

TypeError: unsupported operand type(s) for /=: 'PngImageFile' and 'float'

• [x] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/keras-team/keras.git --upgrade --no-deps

• [x] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

• [x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

[AndrewRodger]

I have run into the same problem. After much mucking around it appears to be in keras/preprocessing/image.py for the module below (its lines 1232 and 1233).

   def _get_batches_of_transformed_samples(self, index_array):
        batch_x = np.zeros((len(index_array),) + self.image_shape, dtype=K.floatx())
        grayscale = self.color_mode == 'grayscale'
        # build batch of image data
        for i, j in enumerate(index_array):
            fname = self.filenames[j]
            img = load_img(os.path.join(self.directory, fname),
                           grayscale=grayscale,
                           target_size=None,
                           interpolation=self.interpolation)
            if self.image_data_generator.preprocessing_function:
                img = self.image_data_generator.preprocessing_function(img)
            if self.target_size is not None:
                width_height_tuple = (self.target_size[1], self.target_size[0])
                if img.size != width_height_tuple:
                    if self.interpolation not in _PIL_INTERPOLATION_METHODS:
                        raise ValueError(
                            'Invalid interpolation method {} specified. Supported '
                            'methods are {}'.format(
                                self.interpolation,
                                ", ".join(_PIL_INTERPOLATION_METHODS.keys())))
                    resample = _PIL_INTERPOLATION_METHODS[self.interpolation]
                    img = img.resize(width_height_tuple, resample)
            x = img_to_array(img, data_format=self.data_format)
            x = self.image_data_generator.random_transform(x)
            x = self.image_data_generator.standardize(x)
            batch_x[i] = x

If it is changed to the following then all works as i expect e.g. the call to self.image_data_generator.preprocessing_function occurs after the image has been resized (if required) and after the image has passed through img_to_array.

   def _get_batches_of_transformed_samples(self, index_array):
        batch_x = np.zeros((len(index_array),) + self.image_shape, dtype=K.floatx())
        grayscale = self.color_mode == 'grayscale'
        # build batch of image data
        for i, j in enumerate(index_array):
            fname = self.filenames[j]
            img = load_img(os.path.join(self.directory, fname),
                           grayscale=grayscale,
                           target_size=None,
                           interpolation=self.interpolation)
            if self.target_size is not None:
                width_height_tuple = (self.target_size[1], self.target_size[0])
                if img.size != width_height_tuple:
                    if self.interpolation not in _PIL_INTERPOLATION_METHODS:
                        raise ValueError(
                            'Invalid interpolation method {} specified. Supported '
                            'methods are {}'.format(
                                self.interpolation,
                                ", ".join(_PIL_INTERPOLATION_METHODS.keys())))
                    resample = _PIL_INTERPOLATION_METHODS[self.interpolation]
                    img = img.resize(width_height_tuple, resample)
            x = img_to_array(img, data_format=self.data_format)
            if self.image_data_generator.preprocessing_function:
                x = self.image_data_generator.preprocessing_function(x)
            x = self.image_data_generator.random_transform(x)
            x = self.image_data_generator.standardize(x)
            batch_x[i] = x

[kmader]

Would it make sense to call the new function postloading_function to offer the ability to perform tasks like random crops and keep the postprocessing_function where it is?

[Vijayabhaskar96]

I guess the changes where not properly reverted to fix this issue, I get this error now!

Epoch 1/10
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py in get(self)
    577             while self.is_running():
--> 578                 inputs = self.queue.get(block=True).get()
    579                 self.queue.task_done()

/usr/lib/python3.6/multiprocessing/pool.py in get(self, timeout)
    643         else:
--> 644             raise self._value
    645 

/usr/lib/python3.6/multiprocessing/pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception)
    118         try:
--> 119             result = (True, func(*args, **kwds))
    120         except Exception as e:

/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py in get_index(uid, i)
    400     """
--> 401     return _SHARED_SEQUENCES[uid][i]
    402 

/usr/local/lib/python3.6/dist-packages/keras/preprocessing/image.py in __getitem__(self, idx)
    826                                        self.batch_size * (idx + 1)]
--> 827         return self._get_batches_of_transformed_samples(index_array)
    828 

/usr/local/lib/python3.6/dist-packages/keras/preprocessing/image.py in _get_batches_of_transformed_samples(self, index_array)
   1233             x = self.image_data_generator.random_transform(x)
-> 1234             x = self.image_data_generator.standardize(x)
   1235             batch_x[i] = x

/usr/local/lib/python3.6/dist-packages/keras/preprocessing/image.py in standardize(self, x)
    583         if self.preprocessing_function:
--> 584             x = self.image_data_generator.preprocessing_function(x)
    585         if self.rescale:

AttributeError: 'ImageDataGenerator' object has no attribute 'image_data_generator'

The above exception was the direct cause of the following exception:

StopIteration                             Traceback (most recent call last)
<ipython-input-18-206f186a346e> in <module>()
      4                                   callbacks=[c1,c2],
      5                                   validation_steps=STEP_SIZE_VALID,
----> 6                                   epochs=10)

/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name +
     90                               '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

/usr/local/lib/python3.6/dist-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
   2190                 batch_index = 0
   2191                 while steps_done < steps_per_epoch:
-> 2192                     generator_output = next(output_generator)
   2193 
   2194                     if not hasattr(generator_output, '__len__'):

/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py in get(self)
    582         except Exception as e:
    583             self.stop()
--> 584             six.raise_from(StopIteration(e), e)
    585 
    586     def _send_sequence(self):

/usr/local/lib/python3.6/dist-packages/six.py in raise_from(value, from_value)

StopIteration: 'ImageDataGenerator' object has no attribute 'image_data_generator'

[Vijayabhaskar96]

Quick temporary fix: pip install git+https://github.com/Vijayabhaskar96/keras.git

[Dref360]

Sorry about that. Should have been right the first time.
Will be fixed there : https://github.com/keras-team/keras/pull/9639

[SauceCat]

quick hack:

def preprocess_input_new(x):
    img = xception.preprocess_input(img_to_array(x))
    return array_to_img(img)

[carsteen]

another quick hack :

```python
def preprocess_fn(im_array):
return im_array/255-.5

class foo(object):
pass

imgen = ImageDataGenerator(preprocessing_function='pass anything')

imgen.image_data_generator = foo
imgen.image_data_generator.preprocessing_function = preprocess_fn

flowgen = imgen.flow_from_directory('/tmp/training_folder/')

flowgen.__getitem__(0)[0] #check result```

[926122009]

I had the same problem and it is fixed since I used TensorFlow version 1.14.0 & Keras version 2.2.4 .