Facenet: validation rate

[zhly0]

the validate_on_lfw.py file output a Validation rate,like: 0.80 +-0.01 @ FAR=0.001.what dose this mean ,is there any related paper or concept?Thanks anyone who can answer this questions .

[luckyboysmith]

hi!
have you understand what this mean?

[davidsandberg]

Hi,
This is related to how the detection threshold is set and the VAL @ FAR=0.001 indicates how many image pairs the detector can correctly identify as the same identity while keeping the false alarm rate (FAR, i.e the probability that two images of different identities are the same) to 0.001. Since the detector need to be more certain, the VAL @ FAR=0.001 is lower than the accuracy.

[luckyboysmith]

@davidsandberg thanks for your reply,it really help me !
Now i have a question,if validation rate will affect accuracy?
i did some experiments two days ago
i align lfw images with size 20X20
and then i imresize the aligned lfw images to size 160X160
the images is blurred but when i run validate_on_lfw.py and set the path to the blurred lfw images
the accuracy is 0.903+-0.01
i am a beginer in deep learning
i think the accuracy is amazing, Was i wrong?

[luckyboysmith]

with validate on the blurred lfw images
the accuracy is 0.903
but valudiation rate is 0.45600

[luckyboysmith]

if validation rate shows how many image pairs the detector can correctly identify as the same identity while keeping the false alarm rate,then what does the accuracy mean?
does it means classify accuracy on lfw datasets?

[luckyboysmith]

what is the relationship between "distance matrix " and "accuracy"?

[luckyboysmith]

@davidsandberg
hope to get your help,pls

[ugtony]

@luckyboysmith,
I did the same experiment as yours before,
I shrink the image size to 5x5, 10x10, 20x20, ... and resize it back to feed into the network.
The accuracy at 20x20 is 0.905+-0.012 and the validation rate is 0.313, which is similar to your results.
So yes, the classifier works even when the input is small (blurred).

[luckyboysmith]

@ugtony
thanks for your reply!
now i am puzzled about the relationship between "validate rate" and "accuracy"
if validation rate shows how many image pairs the detector can correctly identify as the same identity while keeping the false alarm rate,then what does the accuracy mean?
hope to get your help,pls!

[ugtony]

@luckyboysmith ,
If True Positive(TP) is defined as "two faces are validated as the same person, and they are indeed the same", False Negative(FN) is defined as "two faces are judged as different people, but the truth is that they are the same person"
In david's project, "accuracy" means the maximum accuracy (TP+TF)/(TP+TF+FP+FN) under some threshold.
"Val@FAL=0.001" means the rate that faces are successfully accepted (TP/(TP+FN)) when the rate that faces are incorrectly accepted (FP/(TN+FP)) is 0.001.

[luckyboysmith]

@ugtony
your reply really help me a lot,thanks!!!!

[zhly0]

@all,thank you guys,now I think I understand it,the accuracy rate and the validate rate are different,and the are not related,but a better model should have a high accuracy and high validate rate.

[SonVH013]

@ugtony
what is TF mean in (TP+TF)/(TP+TF+FP+FN) ?
i don't understand :D

[ugtony]

@SonVH013,
That was a typo.
It should be TN instead of TF.

[hyderit]

According to the FaceNet paper, FAR means (False Accept Rate) which is the rate of face pairs that are different and yet have a distance that is below a certain threshold (which could be classified as positive).