Glow: Refactor Image Classifier.

[ayermolo]

Right now Image-classifier primary works with well image classification networks. There are other networks such as Segementation, Object Detection. The majority of the code to execute them is the same. Main differences are how some pre-processing stuff and outputs are handled. For example for object detection not only they have multiple outputs, but also usually MAAP score is calculated by python scripts.

Would people be open to idea of refactoring common code in to a seperate file with some hooks for pre-processing and post processing. So that image-classifier, and maybe future apps like object-detection be a wrapper around it?

I am still thinking full implementation through, but wanted to open this to conversation to judge if there is an interest or none starter. Before I invest too much time in to it.

[mciprian13]

I was thinking about this for some while now. I even put some ideas in a document I would like to share:
Glow_Front_End_Tools.zip
I would like to contribute to this. Maybe others would like as well.
Let`s first exchange some ideas: for example you can write your own ideas in the document and maybe send it back.
Cheers!

[ayermolo]

Thanks for sharing your doc. It looks like a more high level approach. I was thinking more low level to provide building blocks for others to build on top of.
More specifically refactor processImageRange, and major functions it's using like loadImagesAndPreprocess, buildAndCompileAndGetInAndOutPair in to a class/seperate file. and provide mechanism to register hooks to

• run on images as pre-process step to be executed on Tensor containing input images.
• run after model is loaded. This can be used to append/prepend graph.
• run custom "processAndPrintResults". So for image classifiers it can be whatever image-classifier does now. For object detection handle multiple outputs to dump if network already has post-processing, or if one desires create it programatically, whatever.

The mechanism I was thinking is having Interface classes that someone can implement and then register with the executor.

I think this should provide flexible enough mechanism to build what you described in your doc.

Just my general idea. Was going to flash it out more next week and write up something.

[mciprian13]

Understood. Your approach is inclined more towards enabling the developers. I was thinking more towards enabling end users directly with an out-of-the-box solution without the need for further development.
We still should see if there are common things we want to do and collaborate on that.

[jfix71]

Seems reasonable to me, but not sure we have cycles to contribute on our end. Happy to review/help land PRs.

[ayermolo]

ImageClassifierRefactorSpec.docx

@jfix71 This is somewhat of a side project for me to cleanup some stuff internally. :)

Anyway let me know what you guys think about the proposal. The Loader extension is somewhat of a preview of what @tlepley wants to upstream.

I think external APIs are generic enough to support various usage models.
Again the scope is limited to primarily abstract some common and nitty gritty details away from app, and allow flexible enough extension mechanism for various usage models.

Also if better names can be suggested by all means let me know. :)

[ayermolo]

@jfix71 @mciprian13 Any feedback?

[jfix71]

@ayermolo Sorry for the delay -- I looked over the doc and it seems reasonable to me. It would be great to see some examples of usage especially of manipulating the graph after loading, and specific opt in command line commands to use. For example to remove the final softmax of e.g. Resnet50 if we are only curious about the top prediction/not worried about the probabilities. Are there any particular extensions you are intending to open source? 🙂

[ayermolo]

@jfix71 Thanks for taking a look, will update the doc with example.
Well after refactor I want to upstream the ObjectDetection app. It uses one of the extension mechanisms to dump multiple outputs in to txt files that then can be fed in to mAP scripts.
There might be another for PP for YoloV2, but I need to double check on that.

[ayermolo]

ImageClassifierRefactorSpec.docx
@jfix71 Added a toy example of registering and implementing Extension.
For your example it will be similar except you will implement LoaderExtension class, and it will go in to postModelLoad. It has input all that is needed. One of it's arguments is an outputMap which will be used in rest of the code.

[ayermolo]

@mciprian13 Submitted MR with my proposed changes. FYI.

[ybai62868]

I think it is great. And I am a new beginner about the glow. Can glow have some nice tutorials？Maybe it can help lots of people to start with the glow? I sincerely hope that the community can give some good examples of how to use glow and to read the codebase of glow.

[ayermolo]

@ybai62868 What kind of documentation would you like that is not already there?
Also as a new user perfect opportunity to contribute if you think something is not clear/missing. :)