Opencv: Failed to load mobilenet-ssd caffemodel.

[oishi89]

Hi there
I'm following this guideline for using opencv dnn android
https://docs.opencv.org/master/d0/d6c/tutorial_dnn_android.html
But I got the issue at net = Dnn.readNetFromCaffe(proto, weights); leading to crash the app.
I made an opencv2.framework for iOS from the latest source code from opencv GitHub and I got the same issue as well.
Do you guys have any idea ? Please help

ex = {InvocationTargetException@5137} "java.lang.reflect.InvocationTargetException"

cause = {UnsatisfiedLinkError@5141} "java.lang.UnsatisfiedLinkError: No implementation found for long org.opencv.dnn.Dnn.readNetFromCaffe_0(java.lang.String, java.lang.String) (tried Java_org_opencv_dnn_Dnn_readNetFromCaffe_10 and Java_org_opencv_dnn_Dnn_readNetFromCaffe_10__Ljava_lang_String_2Ljava_lang_String_2)"

[dkurt]

@oishi89, please check the values of proto and weights.

[oishi89]

@dkurt the proto and weights dowloaded from the link you provided with. I tested with some other models as well. Could you upload those file that you are using?

[dkurt]

@oishi89, I'm sorry but I asked to print actual values of them into logcat after getPath but before readNetFromCaffe.

[oishi89]

[dkurt]

@oishi89, Could you also check version of OpenCV? VERSION string from org.opencv.core.Core package.
And version of OpenCV manager application.

[oishi89]

It's private static String getVersion() { return "3.3.0"; }

[dkurt]

@oishi89, Please check version of OpenCV manager application too. If it's not 3.3.0, you need to update it as described in https://docs.opencv.org/master/da/d2a/tutorial_O4A_SDK.html:

<Android SDK path>/platform-tools/adb install <OpenCV4Android SDK path>/apk/OpenCV_2.4.9_Manager_2.18_armv7a-neon.apk

Note: choose 3.3.0 version and appropriate architecture.

[oishi89]

I have just check the OpenCV manage download from google play store. It's 3.0 so I need to install a 3.3.0 version manually ?

[dkurt]

@oishi89, yeah, exactly. You may find several apk at corresponding folder at unpacked opencv-3.3.0-android-sdk.zip. Install one of them using adb for the target CPU architecture (adb shell cat /proc/cpuinfo may help).

[oishi89]

It's working now.
Thanks for support :)

[oishi89]

@dkurt I turned to a similar sample project on IOS. What I did is making opencv2.framework for iOS from the latest source code from opencv GitHub. Modifying the original https://github.com/opencv/opencv/blob/master/samples/dnn/ssd_mobilenet_object_detection.cpp for feeding camera frames into. It seems that the loading MobilenetSSD model was successful but the failure happened at net.setInput(inputBlob, "data"); Requested blob "data" not found in function setInput. Please take a look into the screenshot below and give me some comment.

There are some confusing that highlighted. cvtColor(image, frame, CV_RGBA2RGB); and inputBlob row and cols = -1

[dkurt]

@oishi89, try to remove input layer name at setInput because actually it has another name.

[oishi89]

I have a confuse between COLOR_RGBA2RGB and CV_RGBA2RGB I tried both but all are failed.

Here is my input

[dkurt]

Am I right that the problem is in .forward call? Is there no error message from OpenCV side? Could you print inputBlob's dims and size[0], size[1] and further?

[oishi89]

Yes the problem is in .forward now and there is no error message form Opencv side until I remove the

color conversion cvtColor(image, frame, CV_RGBA2RGB); then the error message is

OpenCV Error: Unspecified error (Requsted layer "" not found) in getLayerData, file /Volumes/build-storage/build/master_iOS-mac/opencv/modules/dnn/src/dnn.cpp, line 790
libc++abi.dylib: terminating with uncaught exception of type cv::Exception: /Volumes/build-storage/build/master_iOS-mac/opencv/modules/dnn/src/dnn.cpp:790: error: (-2) Requsted layer "" not found in function getLayerData

inputBlob size 1 x 3 x 300 x 300

[dkurt]

@oishi98, Please try to print out net.empty(). Is that true of false?

[oishi89]

I'm using the same MobileNetSSD caffemodel that used in Android sample project.
the net.empty() is always true

[oishi89]

@dkurt Sorry I means the net.empty() is always false :) . We loaded the dnn::Net net = readNetFromCaffe(modelConfiguration, modelBinary); successfully

[dkurt]

@oishi89, May I ask you to do some experiment? Could you open prototxt and call forward with a string parameter of layer names? Up to which layer it works?

[oishi89]

@dkurt Sure ! could you tell me more detail what you want to check.

[dkurt]

Many thanks! We just can find a problem layer. If even forward called up to the first layer produces a crash: the problem is in initialization.

[oishi89]

I tried with conv0, conv1 and detection_out

[dkurt]

@oishi89, that is really looks like an empty network. Can you print layers names from getLayerNames method?

[oishi89]

It looks like the forward called produces a crash at very beginning layer conv0 I'm thinking of a wrong input

[oishi89]

@dkurt

for (int i=0;i<net.getLayerNames().size();i++) {
        cout << "getLayerNames " << net.getLayerNames()[i] << endl;
    }

getLayerNames conv0
getLayerNames conv0/relu
getLayerNames conv1/dw
getLayerNames conv1/dw/relu
getLayerNames conv1
getLayerNames conv1/relu
getLayerNames conv2/dw
getLayerNames conv2/dw/relu
getLayerNames conv2
getLayerNames conv2/relu
getLayerNames conv3/dw
getLayerNames conv3/dw/relu
getLayerNames conv3
getLayerNames conv3/relu
getLayerNames conv4/dw
getLayerNames conv4/dw/relu
getLayerNames conv4
getLayerNames conv4/relu
getLayerNames conv5/dw
getLayerNames conv5/dw/relu
getLayerNames conv5
getLayerNames conv5/relu
getLayerNames conv6/dw
getLayerNames conv6/dw/relu
getLayerNames conv6
getLayerNames conv6/relu
getLayerNames conv7/dw
getLayerNames conv7/dw/relu
getLayerNames conv7
getLayerNames conv7/relu
getLayerNames conv8/dw
getLayerNames conv8/dw/relu
getLayerNames conv8
getLayerNames conv8/relu
getLayerNames conv9/dw
getLayerNames conv9/dw/relu
getLayerNames conv9
getLayerNames conv9/relu
getLayerNames conv10/dw
getLayerNames conv10/dw/relu
getLayerNames conv10
getLayerNames conv10/relu
getLayerNames conv11/dw
getLayerNames conv11/dw/relu
getLayerNames conv11
getLayerNames conv11/relu
getLayerNames conv12/dw
getLayerNames conv12/dw/relu
getLayerNames conv12
getLayerNames conv12/relu
getLayerNames conv13/dw
getLayerNames conv13/dw/relu
getLayerNames conv13
getLayerNames conv13/relu
getLayerNames conv14_1
getLayerNames conv14_1/relu
getLayerNames conv14_2
getLayerNames conv14_2/relu
getLayerNames conv15_1
getLayerNames conv15_1/relu
getLayerNames conv15_2
getLayerNames conv15_2/relu
getLayerNames conv16_1
getLayerNames conv16_1/relu
getLayerNames conv16_2
getLayerNames conv16_2/relu
getLayerNames conv17_1
getLayerNames conv17_1/relu
getLayerNames conv17_2
getLayerNames conv17_2/relu
getLayerNames conv11_mbox_loc
getLayerNames conv11_mbox_loc_perm
getLayerNames conv11_mbox_loc_flat
getLayerNames conv11_mbox_conf
getLayerNames conv11_mbox_conf_perm
getLayerNames conv11_mbox_conf_flat
getLayerNames conv11_mbox_priorbox
getLayerNames conv13_mbox_loc
getLayerNames conv13_mbox_loc_perm
getLayerNames conv13_mbox_loc_flat
getLayerNames conv13_mbox_conf
getLayerNames conv13_mbox_conf_perm
getLayerNames conv13_mbox_conf_flat
getLayerNames conv13_mbox_priorbox
getLayerNames conv14_2_mbox_loc
getLayerNames conv14_2_mbox_loc_perm
getLayerNames conv14_2_mbox_loc_flat
getLayerNames conv14_2_mbox_conf
getLayerNames conv14_2_mbox_conf_perm
getLayerNames conv14_2_mbox_conf_flat
getLayerNames conv14_2_mbox_priorbox
getLayerNames conv15_2_mbox_loc
getLayerNames conv15_2_mbox_loc_perm
getLayerNames conv15_2_mbox_loc_flat
getLayerNames conv15_2_mbox_conf
getLayerNames conv15_2_mbox_conf_perm
getLayerNames conv15_2_mbox_conf_flat
getLayerNames conv15_2_mbox_priorbox
getLayerNames conv16_2_mbox_loc
getLayerNames conv16_2_mbox_loc_perm
getLayerNames conv16_2_mbox_loc_flat
getLayerNames conv16_2_mbox_conf
getLayerNames conv16_2_mbox_conf_perm
getLayerNames conv16_2_mbox_conf_flat
getLayerNames conv16_2_mbox_priorbox
getLayerNames conv17_2_mbox_loc
getLayerNames conv17_2_mbox_loc_perm
getLayerNames conv17_2_mbox_loc_flat
getLayerNames conv17_2_mbox_conf
getLayerNames conv17_2_mbox_conf_perm
getLayerNames conv17_2_mbox_conf_flat
getLayerNames conv17_2_mbox_priorbox
getLayerNames mbox_loc
getLayerNames mbox_conf
getLayerNames mbox_priorbox
getLayerNames mbox_conf_reshape
getLayerNames mbox_conf_softmax
getLayerNames mbox_conf_flatten
getLayerNames detection_out

[dkurt]

@oishi89, Can you create and pass a dummy input blob of size 1x3x300x300 and CV_32F type?
https://docs.opencv.org/master/d3/d63/classcv_1_1Mat.html#a6adc4d64db5379bcbb713e71c9292f6e
And one more question I'd like to ask. Does net.forward("conv0"); produce "Requested layer conv0 not found" or nothing?

[oishi89]

@dkurt OpenCV Error: Unspecified error (Requsted layer "conv0" not found) in getLayerData, file /Users/ln160c/Documents/OpenCV/opencv/modules/dnn/src/dnn.cpp, line 751
I'm making a dummy input blob now. Will update you right away

[oishi89]

@dkurt I highlighted those line of code for creation dummy input please check for sure

[oishi89]

@dkurt If you have any idea please update

[dkurt]

@oishi89, Can you place here a full code or make a reference? Maybe we can find a some bug in the sample.

[oishi89]

@dkurt you can download the project here https://www.dropbox.com/s/qrv0etb351uxq0q/ProjectX.zip?dl=0
This is iOS project running on Xcode 9 please check it out
Thanks

[dkurt]

@oishi89, omg, have you seen that loadModel doesn't return a loaded net? Your global net is empty.

[oishi89]

@dkurt omg... my bad that because of copy/paste. Sorry about that

[dkurt]

@oishi89, no problem :smile: . Please tell if it works now.

[oishi89]

after replacement the local net by global net. It's working perfectly. Sorry for wasting your time.

[dkurt]

@oishi89, never mind. We're always glad to help you!

[4ndre]

@dkurt I know this thread is almost 2 years old, but I have to re-awaken it since I have the very same problem and cannot solve it with the provided details (and it's the only discussion on the problem that I can find). Furthermore, I translated the example simultaneously to Kotlin, but that should not affect the functionality. My app crashes at forward, net is not empty, and I also made it global, but I don't follow fully where the loadModel should be (there is no such function and I assumed it is a descriptive label for readNetfromCaffe or similar?). Hope you guys are still around and can help!

[dkurt]

@4ndre, You may take a look at https://github.com/dkurt/delta9/tree/master/android or newer https://github.com/dkurt/icv_sunday_school_2019_spring/tree/master/classification. As far as I remember in both samples models were loaded to sdcard storage using ADB.

[4ndre]

@dkurt Thanks for the quick response!
Do you mean the getPath function? This one seems to work as it should: model is loaded from app/src/main/assets to /data/user/0/com.APP/files/* ; and it creates a non-empty net. I highlighted this because this was where the discussion went in 2017, but the solution oishi89 came up with is not quite clear. Thought you can elucidate it!

[dkurt]

@oishi89's issue was mostly about code syntax but not deep learning. We initialized local Net instead of global one.

[4ndre]

@dkurt, I tried to dig deeper and net.forward() subfunction yields an exception error:

cv::Exception: OpenCV(3.4.3) /build/3_4_pack-android/opencv/modules/dnn/src/layers/batch_norm_layer.cpp:35: error: (-215:Assertion failed) blobs.size() >= 2 in function 'cv::dnn::BatchNormLayerImpl::BatchNormLayerImpl(const cv::dnn::experimental_dnn_34_v7::LayerParams&)'

I assume almost a syntax error, even though there is nothing obvious to me, so in the worst case it's something stupid.

[4ndre]

@dkurt , From more research I believe the problem could lie in the conf and weight files somehow. Tested a view different combos (even one that works in my python test), but nothing works unfortunately...

[4ndre]

@dkurt I investigated a bit further, and almost rule out the files now, since the net can be imported for working files and fails if it is not ok. I looked what you tried with the blob above, but cannot generate one of size 1x3x300x300. However, checking the blobfromimage size I just get 3x1 back. Here's the code:
`

    val IN_WIDTH = 300
    val IN_HEIGHT = 300
    val WH_RATIO = IN_WIDTH.toFloat() / IN_HEIGHT
    val IN_SCALE_FACTOR = 0.007843
    val MEAN_VAL = 127.5
    val THRESHOLD = 0.2

    mRGB=inputFrame!!.rgba()
    Imgproc.cvtColor(mRGB,mRGB,Imgproc.COLOR_RGBA2RGB)

    val blob = Dnn.blobFromImage(mRGB, IN_SCALE_FACTOR, Size(IN_WIDTH.toDouble(), IN_HEIGHT.toDouble()), Scalar(MEAN_VAL, MEAN_VAL, MEAN_VAL))
   Log.e(TAG,"Blob Size: "+blob.size().toString());`

I'd appreciate any tips, getting to the end of the line here...

[dkurt]

@4ndre, check blob.size(0), blob.size(1), blob.size(2) and blob.size(3) instead

[4ndre]

@dkurt I tried, but I get

Too many arguments for public open fun size(): Size! defined in org.opencv.core.Mat

Thought it's maybe the same reason why my blob creation fails:

val sz=arrayOf<Int>(1,3,300,300)
val iblob=Mat() (4,sz,CvType.CV_32F)

I assumed an outdated version, but I use the openCV343 library. Any ideas?

[dkurt]

@4ndre, Please try newer version or try to debug the code. Looks like this issue does not connected to your problem.

[4ndre]

@dkurt I updated to 3.4.7 and it fixes the Mat problem, which allows me to confirm the size is correct (1x3x300x300). However, it still crashes at forward. I opened a question on stackoverflow to have a broader audience.

[Bambocha]

Hi , I have quite similar problem like @oishi89. I have an error with "OpenCV(3.4.5) Error: Requested object was not found (Layer with requested id=-1 not found) in cv::dnn::experimental_dnn_34_v11::LayerData& cv::dnn::experimental_dnn_34_v11::Net::Impl::getLayerData(int), file /build/3_4_pack-android/opencv/modules/dnn/src/dnn.cpp, " I"m doing my project on Android Studio for Android with this tutorial https://www.youtube.com/watch?v=JasVghcUeyg&list=PLZBN9cDu0MSl6Ei6OzhVfRtSWaxkt0LA1&index=6. I think that can be problem with deprecated function to load file Environment.getExternalStorageDirectory. Any other ideas?