Keras: ImageDataGenerator: image channel issue

[gtuzi]

Using "ImageDataGenerator" on RGB images seems to generate images with non-RGB values. I'm having issues figuring out the mapping to throw the generated images to RGB.

Example:

an RGB image

an image generated from ImageDataGenerator's flow()

I have tried varying the order of the channels but I cannot retrieve a normal image. Has anyone seen this before ?

Keras version: 2.0.1

---

Please make sure that the boxes below are checked before you submit your issue. If your issue is an implementation question, please ask your question on StackOverflow or join the Keras Slack channel and ask there instead of filing a GitHub issue.

Thank you!

• [x] Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

• [ ] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

• [ ] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

• [ ] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

[joelthchao]

It seems like this is not a bug. According to your sample image from ImageDataGenerator 's flow(), there might be following operations apply on your image:

1. Random crop with fill_mode='nearest'
2. Normalization on pixel value

Please provide your code to see what happen

[gtuzi]

Joel, see the following declaration:

traindatagen = ImageDataGenerator(featurewise_center=False,
                                  featurewise_std_normalization=False,
                                  rotation_range=20, width_shift_range=0.3, 
                                  height_shift_range=0.3)

traindatagen.fit(X_train)

...

traindatagen.flow(X_train, y_train, batch_size=2)

[gtuzi]

Joel, just to emphasize, I am concerned with the color channels - I am not getting an RGB image after the image transformation.

[joelthchao]

Nothing wrong happen in my test, not sure if your images saving process has any problem.
Environment:

ENV:
{
    "floatx": "float32",
    "backend": "tensorflow",
    "epsilon": 1e-07,
    "image_dim_ordering": "tf"
}
keras.__version__ == '2.0.1'

Test script:

import numpy as np
from scipy.misc import imread, imsave
from keras.preprocessing.image import ImageDataGenerator

traindatagen = ImageDataGenerator(featurewise_center=False, featurewise_std_normalization=False, rotation_range=20, width_shift_range=0.3, height_shift_range=0.3)

image = imread('test_image.jpg')
images = image[np.newaxis, :, :, :]

process_images = traindatagen.flow(images).next()
process_image = process_images[0, :, :, :]
imsave('test_process_image.jpg', process_image)

[gtuzi]

This probably doesn't matter - but I'm not saving the image. The images are just loaded in memory, transformed and then plotted with plt.

[joelthchao]

Please provides complete code. My concern is the interpretation of channel might different between libraries.

[gtuzi]

import numpy as np
import matplotlib.pyplot as plt
import cv2
images = []

for line in lines:
    sourcepath = line[0]
    filename = sourcepath.split('/')[-1]
    current_path = 'data/IMG/' + filename
    image = cv2.imread(current_path)
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    images.append(image)

X = np.array(images)

image = X[0].squeeze()
plt.figure(figsize=(10,10))
plt.imshow(image)

Here I get:

---

Next:

testgenerator = ImageDataGenerator(featurewise_center=False,featurewise_std_normalization=False,rotation_range=20, width_shift_range=0.3, height_shift_range=0.3)
images = image[np.newaxis, :, :, :]
testgenerator.fit(images)

image = testgenerator.flow(images).next().squeeze()
plt.figure(figsize=(10,10))
plt.imshow(image)

and here I get:

[joelthchao]

Short script to prove that problem not comes from ImageDataGenerator

import numpy as np
from keras.preprocessing.image import ImageDataGenerator

image1 = np.stack([np.zeros((4, 4)), np.ones((4, 4)), np.ones((4, 4)) * 2], axis=-1)
testgenerator = ImageDataGenerator(featurewise_center=False,featurewise_std_normalization=False,rotation_range=20, width_shift_range=0.3, height_shift_range=0.3)
images = image1[np.newaxis, :, :, :]
testgenerator.fit(images)
image2 = testgenerator.flow(images).next().squeeze()

# No channel swap happens
assert np.all(image1 == image2)

[joelthchao]

I understand what happen now, please change your code in the end of script to:

image = testgenerator.flow(images).next().squeeze()
plt.figure(figsize=(10,10))
plt.imshow(image)

# plt interpret value by np.dtype, this will make everything back to normal
image2 = image.astype(np.unit8) 
plt.imshow(image2)

[gtuzi]

That was it! It must be uint8. I was casting to uint16 before this post, and it was giving me the same problem.

[dozyc]

I was casting to int and it gave me the same problem. The colors were actually inverted, if I did plt.imshow(255-img) it would look correct. Using type unit8 fixed it though. Thanks!

[phisad]

Awesome! Thanks for the explanation! Wondered about this a while ago.

[anurag0406]

Where have you added uint8 exacty in your code
Please share, I have a similar error

I was casting to int and it gave me the same problem. The colors were actually inverted, if I did plt.imshow(255-img) it would look correct. Using type unit8 fixed it though. Thanks!