Autokeras: ImageSegmentaion

[chenwydj]

Task Module: Semantic Segmentation (2D)

Task Description

• Input: one image or more, each of them contains one or more objects
• Output: for each input image, each pixel on the image is classified which object or class it belongs to.

Evaluation Metrics: mIoU (mean intersection over union)

1. For each segmentation generated, the foreground IoU is calculated, which is between 0~1
2. Then IoU is computed on a per-class basis and then averaged as mIoU

Benchmark Datasets

1. PASCAL Visual Object Classification (PASCAL VOC) dataset
2. Common Objects in COntext (COCO) dataset
3. Cityscapes

Reason

Semantic Segmentation is one of the most important research areas and applications in Computer Vision, such as autonomous driving, indoor navigation. This module will provide easy access to people or group who needs to perform semantic segmentation in their work with limited Computer Vision or Machine Learning background.

Solution

Some add-ons on current AutoKeras to be considered:

1. Add unpooling layer into search space, and apply restrictions during search (e.g. alignment of pooling/unpooling layers).
2. Consider using some popular architectures as initializations (e.g. UNet, DeepLab).
3. Consider adding complexity constraint (e.g. depth, width) in searching metrics. Currently the accuracy is the only metrics for the model selection.

[DavideBuffelli]

This is very useful, it would be great to have it

[Sandbox3aster]

Interested in working on it

[bhack]

I think that we could not limit this to single image if we have a video source.

A reference dataset:
https://davischallenge.org

Related:
https://github.com/jhfjhfj1/autokeras/issues/84

[bhack]

In future, we plan to incorporate more operations and optimizations into our search space, and apply it to more mobile vision tasks such as semantic segmentation.

https://ai.googleblog.com/2018/08/mnasnet-towards-automating-design-of.html

[blitu12345]

I'm also interested if anyone can guide me how to contribute and work on this?

[haifeng-jin]

@blitu12345 Thank you for you interest.
@chenwydj is the one who is working on this issue.
You may talk to him via our gitter Lobby.
You can find it on our website badges.

Thanks.

[rafaelmarconiramos]

I'm working with image segmentation, and I have a considerable interest in this task.

[bhack]

Related to mnasnet for segmentation:

https://github.com/Shathe/Semantic-Segmentation-Tensorflow-Eager

[bhack]

Searching for Efficient Multi-Scale Architectures for Dense Image Prediction:

https://arxiv.org/abs/1809.04184

[haifeng-jin]

@rafaelmarconiramos Thank you so much for your work.
However, @chenwydj is also working on this one.
You may check with him on Gitter.

Thanks.

[bhack]

Searching for Efficient Multi-Scale Architectures for Dense Image Prediction code just released https://github.com/tensorflow/models/pull/5430

[TusharGupta01]

Can I implement it using FCN?

[chenwydj]

Can I implement it using FCN?

We are currently following the implementation from the ADE20K dataset

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[IamShubhamGupto]

Any updates on the implementation?
I require it as part of my ongoing project, anything i can do to help with the development process ?

[andife]

The project is also interesting for me. What is the status? Is there a possibility to contribute?