Face-api.js: Preloading of the MTCNN model is very slow.

[chtsngn]

Hello,

I'm using VueJS. When you open a Modal, these codes are working, which I created from your sample code. The value of "console.time" is quite high.

Code:

console.time("fullFaceDescriptions");
vm.yuz_degisken_objesi.fullFaceDescriptions = (await faceapi.allFacesMtcnn(vm.yuz_degisken_objesi.videoEl, vm.yuz_degisken_objesi.mtcnnParams))
    .map(fd => fd.forSize(vm.yuz_degisken_objesi.hw.width, vm.yuz_degisken_objesi.hw.height));
console.timeEnd("fullFaceDescriptions");
console.log("*");

fullFaceDescriptions: 5500.3720703125ms
*
fullFaceDescriptions: 372.788818359375ms
*
fullFaceDescriptions: 257.229736328125ms
*
fullFaceDescriptions: 180.324951171875ms
*
fullFaceDescriptions: 222.89404296875ms
*
fullFaceDescriptions: 207.50927734375ms
*

How can I make it more stable?

[justadudewhohacks]

More stable or faster? Faster inference time of the mtcnn detector can be achieved by increasing the minFaceSize parameter for example, which will skip the detection of smaller faces.

[chtsngn]

Preloading? What can be done for this?

fullFaceDescriptions: 5500.3720703125ms

[justadudewhohacks]

Nothing really that I know of, seems like the first time a tensor of a certain shape is uploaded to the GPU there is an initial delay, which they refer to as "warmup" in the mobilenet example. The MTCNN uploads tensors of N + 2 different shapes (N in stage 1, can be adjusted by the maxNumScales forward parameter and 1 each in stages 2 and 3). This unfortunately causes the first forward pass of MTCNN to be much longer on certain machines.

I am not sure where this inital delay comes from (maybe due to allocating textures of certain sizes on the GPU, just a wild guess, I am not that familar with WebGL for GPGPU). If it concerns you, maybe you could ask at tfjs for help.

PS: I would also be interested in the answer to that.

[Panepo]

Here's a comment from a tfjs-example:

Warmup the model. This isn't necessary, but makes the first prediction faster.

This warmup works on computeFaceDescriptor, but not works on mtcnn or allFacesMtcnn on my project. Maybe you can make a predict first after the model is loaded.

[ScottDellinger]

I've been fighting with this for a week or so now... I'm currently looking into web workers as a way to do an initial prediction without tying up the main UI thread so that my actual predictions later on are lightning-fast. I'll let you know if I make any progress.

[ScottDellinger]

Alas... no such luck. I think I'd have to understand tfjs much better than I do to convert it in such a way to get it to work in a web worker. I have gotten it so far as to attempt to load the models, but the registration of the backend failed. Some Googling tells me that webgl isn't supported in Web Workers, but cpu backend also fails to register.

[justadudewhohacks]

Before you are putting too much work into this, I will soon publish some new changes, including a new tiny face detector, which is much faster than ssd and mtcnn and produces much more stable detection results than mtcnn.

I think at that point there will be no real reason anymore to use the mtcnn, unless there is a way to fix the warmup issue.

[chtsngn]

@justadudewhohacks Awesome! I look forward to this.