Icevision: Deployment using Streamlit

[ai-fast-track]

🚀 Feature

Goal: We need a simple deployment example using Streamlit that shows how to use a model that was trained using mantisshrimp library.

Showcase Example: Pets Dataset trained using mantisshrimp. Pytorch model exported and used for inference.

Scenario: App showing a default example with an image with its labels and boxes. Let the user choose an image using one the method described here below, then show the predicted labels and boxes.

First Iteration: features included in the first iteration.
Give the user the following options:
- Option 1: User may choose one of the many default images (examples) provided by the App. This will ease the demo usage by avoiding uploading an image
- Option 2: Give the user the option to upload an image or let the user supply a link to an image
- Once the image supplied, display the corresponding labels and boxes predicted by the model

Second Iteration: Enhancements
1- Once the first iteration is completed, we can add inference statistics by displaying probabilities associated with each box
2- Add the "threshold" as hyper-parameter and let the user play with different threshold values, and update the corresponding boxes. We have to add a control to the App for the threshold parameter.

We also need to figure out which other parameters can be used as App controls to let the user play with different knobs.

[lgvaz]

I'm already working with integrating the pets dataset =)

[oke-aditya]

Great, I am working to get some basic proto of app.

[oke-aditya]

Suggestion, let us keep a seperate branch / repo for demos so that they don't intervene here.

As demo uses mantisshrimp. Mantisshrimp is not dependent on demo.

[lgvaz]

We can have a separate repo (branches don't work well for this) or it can be outside the library folder (like tutorials/examples)

[oke-aditya]

Seperate repo will do the job.
We need to keep the app flexible, so that people can bring in their model.

[oke-aditya]

Development started here

User needs only to specify the stuff in cofig.py file.

Streamlit cannot interactively handle data like upload button, etc. Coz it is not a framework. So this is the workaround.

Once user gives file path etc. we detect the objects and display the O/P.

[oke-aditya]

Facing Heroku Issues. Opened with Streamlit https://github.com/streamlit/streamlit/issues/1636

[oke-aditya]

Connected with Local Deployment Issue #107.

I guess we are most probably gonna run out of SlugSize and other issues on free PaaS platforms.
We have dependencies that will shoot 500 MB easily (pycoco tools, scipy etc we might not think of them but they too create a lot of size). Plus some buildpack issues with Heroku.

Can we show a demo of doing it with IaaS. Like Google Compute Engine, AWS EC2 Free tier. It will be slightly long procedure. We would need to setup firewall access, port forwarding etc. But still possible.

Should help #99

[lgvaz]

Can we try removing those dependencies for inference? For instance, pycocotools is only needed for training, scipy we don't even use, and so on..

[oke-aditya]

We can try to reduce. But note that when we install mantisshrimp itself, we need matplotlib etc which uses scipy internally.
Also without pycoco tools mantisshrimp should work. But it is not working as of now. We can remove that dependency.

[lgvaz]

Can you elaborate why it's not working as of now? I'm talking specifically for inference, I don't think we use it anywhere on predict.

Maybe there are some imports being made that don't allow mantisshrimp to import, is that what you're saying?

[oke-aditya]

It does not work because we do from mantisshrimp.imports import *. And this tries to import all the libraries which causes error. So either we need to do selective imports or we need to do some diffrent API for production.
Also when we mention mantisshrimp in requirments. It will try to install all the requirments file which includes all these meta packages.

[oke-aditya]

I am trying with Streamlit teams. It will be better and easier as their team is more eager to provide support for it. Also less pain to maintain with streamlit. We will try with Heroku / AWS / Google Cloud later.

[oke-aditya]

Deployed over streamlit teams. We are not supposed to share links of our apps for now. But in a few months we will be able to do that.