Opencv4nodejs: New Feature request - Facial emotion recognition

Created on 27 Mar 2018 · 6Comments · Source: justadudewhohacks/opencv4nodejs

Hi,
First of all, great job!!! & thanks a lot.
I would like to use an emotion recognition ( happy / sad ...) ability in my project.
I've detected the face already with the help of your example.
Do you have an example ( or a hint :) ) for implementing emotion detection?
Is there ability for fiducial points recognition, ArUco markers?
Thanks...

Source

YaronHershkovitz

Most helpful comment

@YaronHershkovitz You can use the FER2013 dataset to the train your model. You would only need to write a script that generates landmarks for each face and then use landmarks as features to predict emotions.

https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data

There are features that can be calculated from landmarks that would give you more accuracy. For example, Affectiva uses the following features to train their model:

Brow raise
Brow furrow
Nose wrinkle
Lip corner depressor
Chin raise
Lip pucker
Lip press
Mouth open
Lip suck
Smile

nicholasc on 28 Mar 2018

👍3

All 6 comments

Hm I am not sure if OpenCV is able to do emotion detection, but correct me if I am wrong.

justadudewhohacks on 27 Mar 2018

Thanks for your quick reply.
You are right, Not out of the box.
I thought about using fiducial points, but what I actually intended to do was
finding the face landmark as you call it in your face-recognition.js package and compare the shapes of the mouth for example with a pre-defined shape for emotion recognition.
so I guess i'll use your face-recognition.js for that and opencv4nodejs for the shape comparing & other stuff.
Can I get the mouth points, eyes points... from your package?
Do you think it's a good approach?
Thanks.

YaronHershkovitz on 28 Mar 2018

Yes you can get the 68 point face landmarks with face-recognition.js and you can use the package with opencv4nodejs. Using face landmarks for recognizing face expressions would also be my first guess.

justadudewhohacks on 28 Mar 2018

https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data

There are features that can be calculated from landmarks that would give you more accuracy. For example, Affectiva uses the following features to train their model: