Face-api.js: Improve speed of first detection?

[luisdemarchi]

The first call to detect a photo takes about 10 seconds, and then takes milliseconds for all subsequent detections. I read about the speed in #32 issue, but I'd like to know how to pre-solve some bottleneck that happens only on first detection.

--

Is there any way to call some function to prepare before starting detection and avoid this initial delay? Taking into consideration that the user needs to do an action (click a button) to start the face detection.

I am already doing the initiated boot load. According to the code.

App Init()

const MODEL_URL = "/static/models";
await faceapi.loadSsdMobilenetv1Model(MODEL_URL);
await faceapi.nets.faceLandmark68Net.loadFromUri(MODEL_URL);
await faceapi.loadFaceDetectionModel(MODEL_URL);
await faceapi.loadFaceRecognitionModel(MODEL_URL);

navigator.mediaDevices
  .getUserMedia({ video: { frameRate: { ideal: 10, max: 15 } } })
  .then(stream => {
    this.cameraPreview.srcObject = stream;
    this.cameraPreview.style.display = "block";
  })
  .catch(err => {
    alert("error");
  });

Call Detect

start(){
    configProcessFace();
    detectFace();
}

configProcessFace() {
  this.faceOptions = faceapi.SsdMobilenetv1Options({
    minConfidence: 0.8,
    maxResults: 1
  });
},
async detectFace() {
  const useTinyModel = false;
  const fullFaceDescription = await faceapi
    .detectSingleFace(this.canvasPreview, this.faceOptions)
    .withFaceLandmarks(useTinyModel)
    .withFaceDescriptor();

  if (fullFaceDescription && fullFaceDescription.detection.box) {
    .......
  }
  window.setTimeout(() => {
    this.detectFace();
  }, 400);
},

[justadudewhohacks]

It is actually due to all the shader programs being compiled on the first run when using the WebGL backend.

Is there any way to call some function to prepare before starting detection and avoid this initial delay?

Yes, you can simply call the prediction chain you are using for an empty image or tensor initially, which will compile everything upfront. Just make sure when using a tensor, that the input size matches the input size of the face detector, for the SSD Mobilenet detector it is 512x512.

[luisdemarchi]

@justadudewhohacks Perfect. I push a picture with a face, in the proportions 512x512 and did the recognition while loading the application. When the user will recognize it, it takes 1 second.

For consult:

prepareFaceDetector() {
      let base_image = new Image();
      base_image.src = "/static/img/startFaceDetect.jpg";
      base_image.onload = function() {
        const useTinyModel = true;
        const fullFaceDescription = faceapi
          .detectSingleFace(base_image, new faceapi.TinyFaceDetectorOptions())
          .withFaceLandmarks(useTinyModel)
          .withFaceDescriptor()
          .run()
          .then(res => {
            console.log("--------> " + JSON.stringify(res));
          });
      };
}

[adumbz]

Hi @luisdemarchi , I am facing the same issue for first detection, takes about 12s for me. Would you mind explaining in simpler terms how you made it faster? My code currently looks something like this:

`` const faceMatcher = new faceapi.FaceMatcher(FINALVECTORS,0.5) var idVar = setInterval(async () => { console.log('start capturing intervals') const detections2 = await faceapi.detectSingleFace(video, new faceapi.TinyFaceDetectorOptions({inputSize:128})).withFaceLandmarks().withFaceDescriptor() if (detections2) { const results = faceMatcher.findBestMatch(detections2.descriptor) if (labels.includes(results['label'])) { document.getElementById('msg').innerHTML=(Hello, ${results['label']}!`)
console.log('done')
clearInterval(idVar)
// setTimeout(window.close, 3000)
````

the bulk of the time is spent at at faceapi.detectSingleFace, how did you reduce the time a user would spend here?