Dlib: Batch inference for CNN face detector on Python

[DinoHubber]

Hi, first off, thank you for this wonderful library it has made our lives so much easier. We are looking to perform batch inference for the dlib CNN face detector and are currently scripting with Python, may I know if there are any ways to do this entirely in Python? If not, do we need to dig into the native C++ code?

Thank you!

[davisking]

Yes, just give a list of images as input when you call the detector rather
than one image.

[DinoHubber]

Thanks for getting back. I tried to give a list of images but it didn't work. Here's what I did:

• Detector was created with dlib.cnn_face_detection_model_v1()
• Calling detector(img, 0) for a single image works
• But calling detector(imgs, 0) for a list of images throws "Unsupported image type, must be 8bit gray or RGB image". Specifically, the image list was created by appending k copies of one image into a Python list.
• I also tried tiling the image: imgs = np.tile(img, 10), and got the same exception.

Does the call to the detector require a different collection type?

[davisking]

You should just be able to say detector([img, img]) for instance. That
doesn't work? You might need to update to the new dlib if it doesn't.

[DinoHubber]

Thanks, I upgraded to the latest version of dlib and it worked!

For my understanding, does this method work by performing inference on images one at a time, or does it perform batch inference using the CNN?

[davisking]

Yes, that's what it does. That's the point of this way of calling it.

[DinoHubber]

Thank you for clarifying. No further questions, we can close this issue.

For the benefit of posterity - I upgraded to 19.10.0 to leverage batch inferencing in the CNN face detector. I was previously on 19.09.0, and that didn't work so I assume anything <= that wouldn't work either.

[davidtranno1]

Hi @davisking ,
Many thanks for your great library.

When I run inference in batch mode using a list of image the time for the task is quite long. In concrete, I tried with 4 image in shape (1280, 1280), each image have 6 faces using NVIDIA 2080Ti GPU. If I run detector with upsample_num_times=1 the execution time is approximate 3 seconds, without upsample the time is approximate 1.65 second, I use dlib python.

I think that the slow execution is due to python, it must copy memory to CUDA and back. If i implement my apps using native dlib c/c++ code, is it possible significantly improve execution time for detection? If possible, could you please give me a best practice using dlib for near-realtime object detection with 4 cameras run concurrently.

Thank you.

[davisking]

You are talking about running pretty big images, so it's maybe going to be slow. It's hard to say without really seeing what you are doing. But in any event, there are lots of C++ example programs with detailed reference documentation behind them talking about how to do this. Start with an example program and read the docs for the things it discusses for further details.