Dlib: The order of landmarks have been changed

[pencoa]

I want to train new face landmark detector with another annotation standard.
This is the training data location.

This is the predictor result location.

The predictor is still able to predict the location of face landmarks. But the order of landmarks have been changed. Has anyone met this problem?

The things I have done is that I trained new face detector and modified frontal_face_detector.h and render_face_detection.h in dlib/image_processing.

[davisking]

Loading data correctly is up to you. But if I had to guess I bet you are getting messed up somewhere based on alphabetic sorting of strings (the labels) being different from numeric sorting.

[pencoa]

Oh, you are right. Here is the difference between training xml and example xml.

[sarmadm]

@pencoa , Could you please show me how did you train a new landmark detector ?

[pencoa]

@sarmadm which step?
Firstly, prepare the 'training_with_face_landmarks.xml' with /tools/imglab.
Secondly, training with /python_example/train_shape_predictor.py.

[sarmadm]

for helen data set which contains 2000 images , I need only to detect the eyes .

1. do I have to label the eyes for all the 2000 images ?
2. is labelling ( drawing rectangle ) around the eyes sufficient ?

Thanks

[pencoa]

Step1 draw rectangle with dlib original face detector.
Here is my code.

#!/usr/bin/python

"""
using the original dlib face detector to draw face box
"""

import sys

import dlib
from skimage import io

detector = dlib.get_frontal_face_detector()
win = dlib.image_window()

head = """<?xml version='1.0' encoding='ISO-8859-1'?>
<?xml-stylesheet type='text/xsl' href='image_metadata_stylesheet.xsl'?>
<dataset>
<name>imglab dataset</name>
<comment>Created by imglab tool.</comment>
<images>"""

tail = """</images>
</dataset>"""

print(head)

for f in sys.argv[1:]:
    name = f[-11:]
    img = io.imread(f)
    # detect faces in img
    dets = detector(img, 1)
    if len(dets) < 1:
        continue
    # one face and one box in each image
    box = dets[0]
    # <box top='3' left='154' width='364' height='517'>
    # box.top() box.left() box.width() box.height()
    print("  <image file='%s'>" %name)
    print("    <box top='%d' left='%d' width='%d' height='%d'>"
              %(box.top(), box.left(), box.width(), box.height()))
    print("    </box>")
    print("  </image>")


print(tail)

Step2 extract the landmarks location data in HELEN training dataset and inset them into the xml file from step1. The structure is like my answer above. Pay attention to the order number of each part.

[pencoa]

Of course, you can label directly around the eyes. But you need to train another object detector.
Using the existing face detector is enough for your aim.

[sarmadm]

@pencoa Thanks .

I'm still confused . I run your code like below :
ubuntu@ubuntu:~$ python3.4 train.py

and I got this output :

<?xml version='1.0' encoding='ISO-8859-1'?>
<?xml-stylesheet type='text/xsl' href='image_metadata_stylesheet.xsl'?>
<dataset>
<name>imglab dataset</name>
<comment>Created by imglab tool.</comment>
<images>
</images>
</dataset>

It is not clear for me how can I chose 74 or 194 points for training from the 2000 images . I tried a lot but I didn't have clea idea about it . could you please make it more clear.

[zoha21]

@pencoa do you mind sharing your code for step 2?