Imageai: White and uniform background detection

[nbhupendra]

Hi,
is there a way to detect white background using this? i have image data set for training with white BG and normal.

Really appreciate if you could help me for this.

[rola93]

I'm not sure if this is the right approach... but can you share a sample?

Just from your description, it seems that thresolding techinque and morphological transformations should be enough.

[nbhupendra]

Problem Statement- I need something better approach. Currently i have detected person using face detection and then applied mask on that. Then after on remaining i am using contour detection to find out if image has uniform background or not. Then after using white color range to detect percentage of white color on that. But some time mask does not cover whole body like some one has hair or something on head which comes as contour and alogorithm fail. thats why i need some better option. Like train the model with images and get the result. requirement is something like person should have uniform white background.
self.light_white = np.array([0, 0, 140], dtype=np.uint8)
self.dark_white = np.array([255, 60, 255], dtype=np.uint8)

Another approach- If i could get solution to mask person entire body from image.

Better than this-

x,y,w,h is frontal face detection coordinates.
def create_mask(self, image, x, y, w, h):
mask = np.zeros(image.shape[:2], dtype=np.uint8)
for i in range(0, image.shape[0]):
for j in range(0, max(0, x - self.left_margin)):
mask[i, j] = 255
if i >= min(y + h, int(3 * image.shape[0] / 4)):
mask[i, j] = 0

        k = image.shape[1] - 1
        while k > x + w + self.right_margin:
            mask[i, k] = 255
            if i >= min(y + h, int(3 * image.shape[0] / 4)):
                mask[i, k] = 0
            k -= 1
    return mask

[rola93]

So you have detected the face and would like to segment it between face's pixels and background. Do you need to actually segment the face from background or are you doing it to classify its background later?

[nbhupendra]

i have segmented the face just to find out background color thats all. If it can be done some other way also thats also will help. I just want to check if person has uniform white background or not.

[nbhupendra]

I need verify person's background.
1- It should be uniform
2- it should be white in some range

if there is any object behind person, it should be rejected like passport image.

[rola93]

ok. Then you may try taking an image classification approach. You have two kind (classes/labels) of images: acceptable background and not acceptable background. Just train a classifier to soolve your task. You can check how to do it with image AI here

[nbhupendra]

Ok thank you. so you mean custom object detection training? will it help? Or i should follow some other approach or in case you can guide me or share some samples will help.

[rola93]

No. Just a classifier. Object detection involves more tasks than what you need.

So, your pipeline goes as follows:
1) detect the person's face (you already solve it)
2) cut the detected face, which gives you a new image
3) classify it as useful or not depending on its background, with an image classifier.

To do this last step, check this and this

[nbhupendra]

Thanks but why it is require to cut the face? As you suggested to train using classifier using image ai. If I will cut person face then what about other body part which will still there.

[rola93]

Perhapps I got it wrong, but weren't you interested in the background of the face?
I'm not getting how is your scenario I think

[nbhupendra]

Basically I just want to identify background of the image. So suppose I am clicking a photo of the person standing. If background of the clicked image is not uniform and white then I will throw an error and will ask user to click again if background of the image is white and uniform then it will pass and will return success. Hope it clear your doubt. Do you think if I will train the classifier using image ai on whole image sets will it still work or some other approach? Plz help.

[rola93]

What I still don't get it is how is the original photo. I mean, if it is just a photo of the person with some little background, then go for a classifier, just check the links I already provided on how to do it with imageai.

Otherwise, if your photo contains potentially more than one person you need first to extract the person you are interested in. This also can be achievable with imageai, even without the need of training anything, just a general object detection network will work. Once you get the bounding box you want for the given person, crop it and apply an image classifier on it as described above.

[nbhupendra]

Thats really helpful. I have only single person with some background. So when person comes in then click photo and process that photo for background check if background is uniform and white then allow otherwise show them msg to click again with white background. So as you suggested i will try imageai classifier. Please let me know if thats correct.

i have another query if you could through some light on this. Is there way to identify if image is spoofed or clicked on real person. Like reject the photo if it is photo of photo or video of photo. I have data for this as well for training.

[rola93]

Thats really helpful. I have only single person with some background. So when person comes in then click photo and process that photo for background check if background is uniform and white then allow otherwise show them msg to click again with white background. So as you suggested i will try imageai classifier. Please let me know if thats correct.

Yep, sounds like a plan. Let me know how did you solve it!

i have another query if you could through some light on this. Is there way to identify if image is spoofed or clicked on real person. Like reject the photo if it is photo of photo or video of photo. I have data for this as well for training.

you may try an image classification approach, but not wure how well it may work. It's not that easy. Maybe google it a little bit before trying anything.

maybe @OlafenwaMoses should close this issue as the main doubts with respect to ImageAI were already answered. Feel free to reopen it if help is needed

[nbhupendra]

Thank you @rola93 . Really appreciate. Will train and update you outcome.

[nbhupendra]

works like charm. I get the accuracy 92.76 with the resnet model with 20 exp after that 92 continuous (used 1100 images with white bg and 1100 with non white. and 1000 with test data). now let me test this with real images. Thanks @rola93 @OlafenwaMoses . But when i am trying to train with densenet which is more accurate i am getting accuracy 70 with 10 experoment and after that its dropping? please guide me if i may have to do so something to achieve better accuracy as densenet is more accurate i guess as per your document.

[OlafenwaMoses]

Have you tried more than 20 (e.g 50 or more) for DenseNet ?

[rola93]

I'm a little lost on what point are you refering to. You are trying to classify the just the iamges with the numbers?

A good insight on what is happenng with your model, is to see how de loss function evolves with training, this gives lot of clues on how is everything going.

As a side note for future questions like this that are not exactly issues, but discussions (which are pretty interesting too), there is a new forum to discuss that @OlafenwaMoses recently launched

[nbhupendra]

@olafenwaMoses no I have not tried with 50 exp as loss was increasing after 5 itself and accuracy came down from 70 to 50 and onwards continued 50 on every experiment. But it could suggest I will train that again with densnet with 50 or more and will see if loss comes down or accuracy increases further.

@rola93 loss was increasing in every steps in dense net but in resent where actually I hit 93% accuracy actually loss was coming down in every exp that’s why I achieved that accuracy that means models was learning in every steps I guess. I am not that much of expert. But if I did any thing wrong plz let me know. Yes I am doing image classification only like I said earlier to predict images which one is with white background or not.

Also thanks for Starting forum It will help us to discuss solution for problems.

[rola93]

I don't know every detail on the image classification API of ImageAI, (I've been using/contributing to object detection), so I'll share some thoughts which you should contrast with docs.

More samples is always a possible answer. It's well known that you can't always have more samples and that it is an expensive resource. One thing you can try is data augmentation: that is: use your examples to generate diferent variations of it: try flipping the images horizontally for instance. I'm not sure if this option is currently available on ImageAI, (it could, since keras allows to fit generators of images out the box), but you can always generate them before hand increase your training dataset.

Another point with respect to the evolution of the error/loss, is the following: some times is not that important to see if it is increasing/decreasing, but how fast it is changing. Plotting them is the best way to see it. I wrote a PR adding logging strategies for TensorBoard which would allow you to see its beheaviour (even while training) but I'm not sure if it is available con image classification. Once more: checkout the docs to be sure