Darknet: Face Classification after Face detection

[dexception]

I am looking to train the model for the following in Yolo. Since i don't want separate projects for same video processing. So after face detection ..do i need to use opencv for the expression classification ?

https://github.com/oarriaga/face_classification

Can this be done in Yolo ?

[AlexeyAB]

You can try to train yolov3.cfg on fer2013/IMDB datasets to detect faces in a such way that each class_id corresponds to different emotion. You should convert labels to Yolo format by yourself.

Yolo v3 should have higher accuracy than: https://github.com/oarriaga/face_classification/blob/master/report.pdf

Our first model
relies on the idea of eliminating completely the fully connected
layers. The second architecture combines the deletion of the
fully connected layer and the inclusion of the combined
depth-wise separable convolutions and residual modules. Both
architectures were trained with the ADAM optimizer [8].

[dexception]

@AlexeyAB
Is it possible to merge single class models with each other ? Can we accomplish this with a command line option to merge models ?

[AlexeyAB]

@dexception No, it can't be done.

[dexception]

@AlexeyAB
What about multi labeling an object ? Can i mark a single face in an image under the following 5 categories ?

face,person,male,white-caucasian,happy

[AlexeyAB]

@dexception Yes for Yolo v3 (but not for Yolo v2).
Just add several labels to the txt-file with the same coordinates but with different class_id.

So, if deltas for classes was already calculated, then next truth_class_id for the same final cell (x,y,anchor) will calculate only its own delta: https://github.com/AlexeyAB/darknet/blob/6390a5a2ab61a0bdf6f1a9a6b4a739c16b36e0d7/src/yolo_layer.c#L112-L119

As you can see delta_yolo_box for the same final cell (x,y,anchor) and for different class_id's will be overwritten (and if different class_id's have the same coordinates - then it doesn't matter).
But delta_yolo_class for the same final cell (x,y,anchor) and for different class_id's will be concatenated: https://github.com/AlexeyAB/darknet/blob/6390a5a2ab61a0bdf6f1a9a6b4a739c16b36e0d7/src/yolo_layer.c#L234-L277

For the 1st class_id:

        for (n = 0; n < classes; ++n) {
            delta[index + stride*n] = ((n == class_id) ? 1 : 0) - output[index + stride*n];
            if (n == class_id && avg_cat) *avg_cat += output[index + stride*n];
        }

for the rest class_id's:

    if (delta[index + stride*class_id]){
        delta[index + stride*class_id] = 1 - output[index + stride*class_id];
        if(avg_cat) *avg_cat += output[index + stride*class_id];
        return;
    }

[dexception]

@AlexeyAB
So for image1.jpg

face,person,male,white-caucasian,happy as one category in obj.names file
(Obviously not but want to be sure)
and
0 0.xx 0.xx 0.xx 0.xx
in image1.txt file ?

or

face
person
male
white-caucasion
happy
all as separate categories in obj.names file
and
0,1,2,3,4 0.xx 0.xx 0.xx 0.xx
in image1.txt file

I am asking this because of the size of dataset i would have to prepare.

[AlexeyAB]

@dexception

Multilabel training by using such files:

image1.txt - labels

0 0.2 0.3 0.1 0.1
1 0.2 0.3 0.1 0.1
2 0.2 0.3 0.1 0.1
3 0.2 0.3 0.1 0.1

---

obj.names - names of classes

face
person
male
white-caucasion

[lynnw123]

How about in the prediction stage, will it prefer to predict the bounding box as "face" only rather than "male", or "white-caucasian", or "happy", considering there are more faces labels than others? Is there another way (a tree structure?) that would guarantee one of each gender/ethnicity/emotion categories to be predicted?

[AlexeyAB]

@Vmaxx-yinling

• Yolo v3 will predict all classes for the same result bounded box, so many classes can be more than 90% of probability, because there is used sigmoid logistic activation instead of softmax. There is no tree structure in the Yolo v3. After this, you can manualy build tree in your C/Python code and select preferable branch of tree.

• Tree structure is used inside the Yolo 9000, but it has lower accuracy: https://github.com/AlexeyAB/darknet#using-yolo9000

[EtheneXiang]

Multilabel training

how to modify my cfy file for Multilabel training ??

[EtheneXiang]

@dexception

Multilabel training by using such files:

image1.txt - labels

0 0.2 0.3 0.1 0.1
1 0.2 0.3 0.1 0.1
2 0.2 0.3 0.1 0.1
3 0.2 0.3 0.1 0.1

obj.names - names of classes

face
person
male
white-caucasion

@AlexeyAB how to modify my cfy file for Multilabel training, i have known how to modify the txt file(label file for training)for Multilabel , shall i change the yolo layer in my cfg file???

[AlexeyAB]

@EtheneXiang You shouldn't modify cfg-file. yolov3.cfg ... yolov3-spp.cfg supports multilabeling by default.

[EtheneXiang]

@EtheneXiang You shouldn't modify cfg-file. yolov3.cfg ... yolov3-spp.cfg supports multilabeling by default.

how about darknet53.cfg, dose it support multilabeling classfication by default??? if it does not support,how to modify for multilabeling classfication? THX very very much@AlexeyAB

the last two lines in darnet53.cfg are:
[softmax]
groups=1

[EtheneXiang]

@EtheneXiang You shouldn't modify cfg-file. yolov3.cfg ... yolov3-spp.cfg supports multilabeling by default.

and another question : how to generate labels for multilabeling classfication??
in offical example,for one lable-classfication, the label format is each image's label in its image filename,like this : image1_mylabel1.jpg image2_mylabel2.jpg.
but in multilabeling classfication, i do not know the label format, could you tell me, THX very much @AlexeyAB

[sagarkar10]

@EtheneXiang Did you find answers to the above questions?

[EtheneXiang]

@EtheneXiang Did you find answers to the above questions?

no i just konw how to use yolov3 for multi-label detection.

[xjohnxjohn]

@EtheneXiang you see this post https://github.com/AlexeyAB/darknet/pull/3728

[EtheneXiang]

@EtheneXiang you see this post #3728

great, thx