Face-api.js: Unable to integrate it with react native

[sedaplaksa]

tried to integrate the face api in react native but tfjs-image-recognition is in fs format. is there any way to resolve issue. Really need help as this is a school project . Thanks

[justadudewhohacks]

What is a fs format? Can describe the issue more precisely?

[sedaplaksa]

when i did an npm i face-api.js and import * as faceapi from 'face-api.js' into my page , it threw me an error fs module does not exist in module haste map

Did a check online and found the tjfs image recognition which is in fs format can only be work in web browser.

So im wondering is that anyway I can solve this error and make it run on mobile? I really need help on these issues . thanks.

[justadudewhohacks]

Did a check online and found the tjfs image recognition which is in fs format can only be work in web browser.

I am maintaing tfjs-image-recognition base, but I still don't know what "fs format" should be. A more detailed error message would be helpful.

[sedaplaksa]

Just a quick ask is it possible for faceapi to be used on react native currently?

[sedaplaksa]

I suspect this maybe where the error come from since react native cannot read 'fs' file format

[wulforr]

I too am facing the same problem in React-Native
I have installed face-api.js and then imported it as import * as faceapi from 'face-api.js';
Then I get the following errors

 bundling failed: Error: Unable to resolve module `fs` from `node_modules/tfjs-image-recognition-base/build/commonjs/env/createFileSystem.js`: fs could not be found within the project.

If you are sure the module exists, try these steps:
 1. Clear watchman watches: watchman watch-del-all
 2. Delete node_modules: rm -rf node_modules and run yarn install
 3. Reset Metro's cache: yarn start --reset-cache
 4. Remove the cache: rm -rf /tmp/metro-*
    at ModuleResolver.resolveDependency (/root/React-native/faceapi/node_modules/metro/src/node-haste/DependencyGraph/ModuleResolution.js:186:15)
    at ResolutionRequest.resolveDependency (/root/React-native/faceapi/node_modules/metro/src/node-haste/DependencyGraph/ResolutionRequest.js:52:18)
    at DependencyGraph.resolveDependency (/root/React-native/faceapi/node_modules/metro/src/node-haste/DependencyGraph.js:282:16)
    at Object.resolve (/root/React-native/faceapi/node_modules/metro/src/lib/transformHelpers.js:267:42)
    at /root/React-native/faceapi/node_modules/metro/src/DeltaBundler/traverseDependencies.js:426:31
    at Array.map (<anonymous>)
    at resolveDependencies (/root/React-native/faceapi/node_modules/metro/src/DeltaBundler/traverseDependencies.js:423:18)
    at /root/React-native/faceapi/node_modules/metro/src/DeltaBundler/traverseDependencies.js:275:33
    at Generator.next (<anonymous>)
    at asyncGeneratorStep (/root/React-native/faceapi/node_modules/metro/src/DeltaBundler/traverseDependencies.js:87:24)

[lacson17]

Hi @sedaplaksa @wulforr,

It could very much be possible that tfjs-image-recognition-base is currently incompatible with React Native...

Node is only used by the packager to serve/compile your app bundle; in other words, React Native apps don't run in the node environment.

"fs" stands for file system, and in order to access it in React Native, you would have to use something like react-native-fs or rn-nodeify in order to bridge and talk to each platform's native APIs (the iOS/Android platforms, in this case). If you could find a way to integrate these into tfjs-image-recognition-base for your specific app use-case, then it should work.

@justadudewhohacks please correct me if I'm wrong in anything that I've stated above.

[msvargas]

I think its posible fix, you add 'fs empty' in webpack and create custom fetch in faceapi.env.monkeyPatch, but i dont know if this working in react-native

[sedaplaksa]

Hi @sedaplaksa @wulforr,

It could very much be possible that tfjs-image-recognition-base is currently incompatible with React Native...

Node is only used by the packager to serve/compile your app bundle; in other words, React Native apps don't run in the node environment.

"fs" stands for file system, and in order to access it in React Native, you would have to use something like react-native-fs or rn-nodeify in order to bridge and talk to each platform's native APIs (the iOS/Android platforms, in this case). If you could find a way to integrate these into tfjs-image-recognition-base for your specific app use-case, then it should work.

@justadudewhohacks please correct me if I'm wrong in anything that I've stated above.

i tried this method but it still did not work sadly :(

[sedaplaksa]

Hi does anyone have any other different solutions? Or is it just not compatible at the moment ?

[wulforr]

Well i removed face-api from react native and used it in node and sent my request from react native to node.

[sedaplaksa]

Do u mind sharing with me how u did it? Is there any latency during detection ?

[deividsoncs]

Hi guys, @wulforr can you give more details of you solution, im with the same problem, but i can turn it to server-side, i have to solve this issue on mobile app.

[artem-kushal]

Did anyone try integrate face api with this adapter?https://github.com/tensorflow/tfjs/tree/master/tfjs-react-native

[deividsoncs]

Did anyone try integrate face api with this adapter?https://github.com/tensorflow/tfjs/tree/master/tfjs-react-native

@artem-kushal may be one way to do this work, i didn't konw tfjs-react-native, i will try use it, please comment bellow if you got success...

[wulforr]

Hi guys, @wulforr can you give more details of you solution, im with the same problem, but i can turn it to server-side, i have to solve this issue on mobile app.

Sure @deividsoncs,
I used face-api.js to find similarities between two images. I had images stored in the database and sent my image from mobile app to server.
In server this image is compared to all the images in the database.

If you wanna know more or anything in specific do let me know.

[deividsoncs]

Hi guys, @wulforr can you give more details of you solution, im with the same problem, but i can turn it to server-side, i have to solve this issue on mobile app.

Sure @deividsoncs,
I used face-api.js to find similarities between two images. I had images stored in the database and sent my image from mobile app to server.
In server this image is compared to all the images in the database.

If you wanna know more or anything in specific do let me know.

Thanks @wulforr , but my problem is that a have to solve in a mobile app to send only the features extracted from the image captured in a mobile camera, because the image is taking so long time to uploado to the servers, im live at Brazil and the mobile internet connection is so slow in many places here. So, i wish do this at app, could you help on this?

[LauraPeraltaV85]

Hi guys, @wulforr can you give more details of you solution, im with the same problem, but i can turn it to server-side, i have to solve this issue on mobile app.

Sure @deividsoncs,
I used face-api.js to find similarities between two images. I had images stored in the database and sent my image from mobile app to server.
In server this image is compared to all the images in the database.

If you wanna know more or anything in specific do let me know.

Hi @wulforr, by any chance is your repo of this project public, if yes can you please tell me the name to check it out, I have been looking for examples of how to use face-api.js and I'm a little confused, thank you

[ibaam97]

I assume the library can be implemented in a react native webview

[CHOMNANP]

If anyone manage to make face-api work with react-native, please share your solution.
I'm trying to figure out as well.

[yasahmed]

use https://www.npmjs.com/package/react-native-fs instead of fs

[sandys]

@yasahmed have you got face-api.js to work with react native ? can you post an example ?

[0xori]

as said before, you can use https://www.npmjs.com/package/react-native-fs instead of fs, but it's not enough.
you'll need to install https://www.npmjs.com/package/@tensorflow/tfjs-react-native.
use faceapi.setEnv to initialize, face-api is excpecting an Environment interface

{
  Canvas: typeof HTMLCanvasElement
  CanvasRenderingContext2D: typeof CanvasRenderingContext2D
  Image: typeof HTMLImageElement
  ImageData: typeof ImageData
  Video: typeof HTMLVideoElement
  createCanvasElement: () => HTMLCanvasElement
  createImageElement: () => HTMLImageElement
  fetch: (url: string, init?: RequestInit) => Promise<Response>
}

I was using react-native-canvas for the canvas elements
here is a partial implementation

var RNFS = require('react-native-fs');
import Canvas, {
  Image,
  ImageData,
  CanvasRenderingContext2D,
} from 'react-native-canvas';
import * as faceapi from 'face-api';
import { cameraWithTensors } from '@tensorflow/tfjs-react-native';
import { Camera } from 'expo-camera';
const TensorCamera = cameraWithTensors(Camera);


const createCanvasElement = function () {
  if (Canvas) {
    return new Canvas();
  }
  throw new Error(
    'createCanvasElement - missing Canvas implementation for RN environment',
  );
};

const createImageElement = function () {
  if (Image) {
    return new Image();
  }
  throw new Error(
    'createImageElement - missing Image implementation for RN environment',
  );
};

const readFile = function (filePath: string) {
  try {
    return RNFS.readFile(filePath);
  } catch (e) {
    console.error('error reading file', e);
  }
};

const fileSystem = {
  readFile,
};

const _fetch =
  fetch ||
  function () {
    throw new Error(
      'fetch - missing fetch implementation for browser environment',
    );
  };

 let env = {
    Canvas: Canvas,
    CanvasRenderingContext2D: CanvasRenderingContext2D,
    Image: Image,
    ImageData: ImageData,
    Video: TensorCamera,
    createCanvasElement,
    createImageElement,
    fetch: _fetch,
    ...fileSystem,
  };

  faceapi.env.setEnv(env);

for loading the models i uploaded them to external storage (AWS S3)
and used await faceapi.nets.tinyFaceDetector.loadFromUri('https://your-bucket-urls3.amazonaws.com/models');

and I used this example to get a proper input that I can pass to the faceDetector: https://github.com/tensorflow/tfjs/blob/master/tfjs-react-native/integration_rn59/components/webcam/realtime_demo.tsx

Everything haw worked without any errors but I couldn't get any results, also the detection process was very slow compared to web browser.

Good Luck, and please let us know if you have any breakthrough

[gravestar]

@0xori im trying to do step that you mention, but im still getting error, did u manage to finish that code ?