Cameraview: Integration with firebase MLKIT

Created on 20 Dec 2018 · 11Comments · Source: natario1/CameraView

An integration guide with firebase MLKIT

Hello Everyone, I am working on a project that uses the firebase machine learning framework MLKIT for an application that uses a barcode reader. However, the cameraView proposed by the framework is terrible. It doesn't work on all devices.

Because I am not that familiar with CameraView. i wonder if someone can write a guide on how to integrate it with MLKIT barcode scanner.

Discussion on mlkit repo : https://github.com/firebase/quickstart-android/issues/745

Tnak you in advanced.

question

Source

g-rabah

All 11 comments

I'm not a MLKIT expert so I'll leave this one to someone else. You will need to use a FrameProcessor in CameraView and pass data to ML

natario1 on 20 Dec 2018

👍1

@sshrabah CameraView makes it incredible easy to work with MLKit due to its FrameProcessor.
My integration guide would be something like:

Set the minimal requirements for CameraView.
Set a Frame Processor
Use the FrameProcessor to pass the frames to MLKit

Just make sure to follow up on Performance Tips from MLKit.

I built an OCR App succesfully with Camera2 before finding out about CameraView, with CameraView I get half the lines of code(and out of it 90% is code unrelated to de CameraView, 8 lines to set the CameraView) than with Camera2, and I am newbie.

@natario1 It is my understanding that using the FrameProcessor to process anything like MLKit is not being done in the Main Thread, is that correct?

GuanacoDevs on 20 Dec 2018

👍1

Correct!

natario1 on 20 Dec 2018

❤1

Hey @natario1 I have developed face detection using ML Kit and Camera preview frame processing using FrameProcesser. But as and when I start video capturing, Frame processor callback stopped. I am not sure why it is stopped, Is that intentionally or It is due to some incorrect setup at my side?

bipinvaylu on 1 Jan 2019

This is how Camera1 works, sorry. You can't receive callbacks while capturing video. In v2 you will be able to capture a videoSnapshot which does not block the frame processor.

natario1 on 3 Jan 2019

Great Thanks, @natario1 for your active contribution and responses. Is videoSnapshot working on v2 branch? As I had cloned your project because v2 beta gradle integration is not working. It would be a great help if you can provide some workaround.

bipinvaylu on 4 Jan 2019

I actually succeeded to do what I wanted using FrameProcessor. This thing rocks. thanks @natario1

g-rabah on 4 Jan 2019

👍1

Cool! Yes, videoSnapshot should be working.

natario1 on 4 Jan 2019

Hey @sshrabah, It would be nice if you can share your thoughts on how you figure it out. I am wondering if you haven't faced this issue?

bipinvaylu on 4 Jan 2019

how to implement it,, for smiling face detection..can you please any sample ?

Priyanka-Mohanty on 13 Mar 2019

@sshrabah

Is your ML Kit working fine with camera View on All devices.
My Bounding box is getting shifted to the left in some of the devices while others it works fine. Im already using frame processor .

Can you help me with this if you have faced the same issue?
also @natario1 can you suggest something on this