Darknet: A better or easy way to convert YOLO v4 tiny and full models to mobile devices (iOS and android)

[tobyglei]

I googled around and looked for some of the best ways to deploy the YOLO models on mobile devices, but I couldn't find a good one.

The recommend or most popular way to do that is to convert the darknet weights to tensorflow / pytorch weights, then use coremltools to do the weight conversion to support iOS. However, there are many unsupport layer or functions on CoreML so it is not as easy as I thought to convert the weights between different platforms.

Hope to see if we can get the pretrained weight on mobile devices and generate a FPS comparison chart between different Mobile chips(e.g., Apple A series).

I can help out if you find this is an interesting direction for this project.

[AlexeyAB]

• You can try to train model in Darknet: https://github.com/AlexeyAB/darknet

• Then convert this yolov4.cfg/yolov4.weights files to yolov4.tf TensorFlow graph-model, and then to yolov4.tflite TFLite-model by using code: https://github.com/hunglc007/tensorflow-yolov4-tflite

• Then modify these examples and use yolov4.tflite model:

  • Android: https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection/android
  • iOS: https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection/ios
  • RPi3: https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection/raspberry_pi

---

Another way to install OpenCV on Smartphone and use yolov4.cfg/yolov4.weights directly by using OpenCV on iOS/Android: https://opencv.org/releases/

• C++ https://github.com/opencv/opencv/blob/master/samples/dnn/object_detection.cpp
• Python https://github.com/opencv/opencv/blob/master/samples/dnn/object_detection.py
• Description: https://docs.opencv.org/master/da/d9d/tutorial_dnn_yolo.html

---

Another way to use Tencent/NCNN library to run yolov4: https://github.com/Tencent/ncnn

[tobyglei]

Thanks for the detail response. I will give it a try.

[fm64hylian]

there is also the option of converting to keras model and then to mlmodel: https://github.com/Ma-Dan/YOLOv3-CoreML
or you can use this guide to add the mish layer on convert.py for yolov4 (or replace convert.py with this script and it will convert directly to mlmodel)

then you convert your .h5 keras model with this repository, so in short

convert weights to keras
python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5

then convert keras to mlmodel on the /Convert folder from this repository
python coreml.py

OR use the script
python3 convert.py yolov4.cfg yolov4.weights yolov4.mlmodel

I could implement my custom yolov3 model on IOS using those repositories , I haven't tested the generated YOLOv4 mlmodel on the xcode side because I don't own a macOS but, and if someone know how please let me know, the input and output parameters when converting to mlmodel that the pretrained apple mlmodel offer, the input and output parameters are different. So if someone knows how to modify the parameters when converting to mlmodel or keras, so that the outputs are confidence and iouThreshold instead of whatever output1, output2 and output3 means (it says "The 13x13 grid (Scale1)", 2626 scale2, but I don't really know what that means), that would be great.

Thank you.

[syedmustafan]

there is also the option of converting to keras model and then to mlmodel: https://github.com/Ma-Dan/YOLOv3-CoreML
or you can use this guide to add the mish layer on convert.py for yolov4 (or replace convert.py with this script and it will convert directly to mlmodel)

then you convert your .h5 keras model with this repository, so in short

convert weights to keras
python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5

then convert keras to mlmodel on the /Convert folder from this repository
python coreml.py

OR use the script
python3 convert.py yolov4.cfg yolov4.weights yolov4.mlmodel

I could implement my custom yolov3 model on IOS using those repositories , I haven't tested the generated YOLOv4 mlmodel on the xcode side because I don't own a macOS but, and if someone know how please let me know, the input and output parameters when converting to mlmodel that the pretrained apple mlmodel offer, the input and output parameters are different. So if someone knows how to modify the parameters when converting to mlmodel or keras, so that the outputs are confidence and iouThreshold instead of whatever output1, output2 and output3 means (it says "The 13x13 grid (Scale1)", 2626 scale2, but I don't really know what that means), that would be great.

Thank you.

The same issue I am facing too. I want to get the output as confidence and coordinates. Anyone kindly suggest a method to implement that.

[vak]

@AlexeyAB thank you for providing yolov4-tiny.weights on the README.md and the reference to tensorflow-yolov4-tflite on how to get YoloV4 on TF-Lite!

Unfortunately converting your yolov4-tiny.weights to TF-Lite using tensorflow-yolov4-tflite doesn't work:

python save_model.py --weights ./data/yolov4-tiny.weights --output ./checkpoints/yolov4-tiny-416-tflite --input_size 416 --model yolov4 --framework tflite

causes:

  File "/home/dev/tensorflow-yolov4-tflite/core/utils.py", line 63, in load_weights
    conv_weights = conv_weights.reshape(conv_shape).transpose([2, 3, 1, 0])
ValueError: cannot reshape array of size 554878 into shape (256,256,3,3)

Also @AlexeyAB you mentioned that you convert to the TF-Lite in your own way to avoids the garbage in output. Is this converter open?

Or maybe it is possible to share yolov4-tiny.tflite exported by your converter?

[AlexeyAB]

@vak

1. You forgot --tiny flag

2. Try
   python save_model.py --weights ./data/yolov4-tiny.weights --output ./checkpoints/yolov4-tiny-416-tflite --input_size 416 --model yolov4 --framework tflite --tiny

3. There is garbadge only when using universal converters like pytorch -> onnx -> pb -> tflite. But if you are using the native YOLOv4 implementation on TensorFlow, then there should be no garbage.

Also @AlexeyAB you mentioned that you convert to the TF-Lite in your own way to avoids the garbage in output. Is this converter open?

1. This converter is private. And this is for Pytorch->TFlite only.

---

There is converter Darknet -> ONNX: https://github.com/linghu8812/tensorrt_inference/tree/master/Yolov4

• it works well: https://github.com/AlexeyAB/darknet/issues/7002

There is converter OpenVINO -> PB -> TFlite: https://github.com/PINTO0309/openvino2tensorflow

• It works well for MiDaS (monocular-depth estimation): https://github.com/PINTO0309/PINTO_model_zoo/issues/15#issuecomment-714441709

• And maybe it works well for YOLOv4 (I did not test it well): https://github.com/PINTO0309/PINTO_model_zoo/issues/15#issuecomment-731655178