Tfjs: Feature Request - Deeplab/MobileNet application or example

[dhrumil83]

To get help from the community, we encourage using Stack Overflow and the tensorflow.js tag.

TensorFlow.js version

1.x

Browser version

Chrome

Describe the problem or feature request

TFLite has this section on segmentation. Something similar optimized for browser would be awesome.
https://www.tensorflow.org/lite/models/segmentation/overview

Code to reproduce the bug / link to feature request

[sdll]

I would love to take this issue on as part of my GSoC 2019 project, consulting the standard implementation as a reference and using the mobilenetv2_coco_voctrainaug pre-trained model in the backend. Please let me know if there are any points to consider beforehand.

[dsmilkov]

Using the mobilenet architecture as the backbone sounds great since this needs to be an edge/mobile-friendly model. I don't have any concrete points, but happy to provide guidance as you go further along.

[sdll]

A little update: DeepLab is now on its way to the official launch (see https://github.com/tensorflow/tfjs-models/pull/229).

[dhrumil83]

Awesome. Thanks @sdll
Looking forward to the official launch.

Hoping to achieve this workflow in the browser:
If you have some imagery data and you want to extract spatial features from it, currently GIS analysts manually digitize the features on the map. e.g. Drawing an outline for a building or lake
What if you can click on a building or lake and it can detect and digitize it into features.

[sdll]

@dhrumil83, this is great! Sounds like you will need to fine-tune DeepLab and then convert the weights yourself. Have you tried that already?

[dhrumil83]

@sdll Not yet. Will start looking into the training part in the next couple of weeks.