Insightface: Threshold

[orlevit]

Hi,
Could you please provide the threshold used in $INSIGHTFACE/src/eval/verification.py
(The one that you used in your paper for test)

Thanks!

[nttstar]

Threshold is not a fixed value.

[orlevit]

Why it is not fixed?
When testing a data set, shouldn't it be a fixed (Learned hyperparameter getting from the training)

[ltkhang]

Why it is not fixed?
When testing a data set, shouldn't it be a fixed (Learned hyperparameter getting from the training)

Of course, the threshold is not fixed.
For example you have MS1M dataset, there are 2 ways to find the threshold

1. You train all of the things and find a threshold on the train-set itself in a statistical way.
2. you split you set into train and val set, then apply a statistical method on val-set to find the best threshold.
   Finally, you can apply this threshold value on public test set such as LFW, Megaface, CFP, etc.

[orlevit]

@ltkhang
Thanks for the answering, but I was referring to the test set.
Which thresholds have been used in the test sets(In the test sets it should be the one that got from the validation set, isn't it)

[ltkhang]

@ltkhang
Thanks for the answering, but I was referring to the test set.
Which thresholds have been used in the test sets(In the test sets it should be the one that got from the validation set, isn't it)

no, the test set is independent from the validation set.
you can imagine that, train set and validation set are the things you know and the test set is the thing you don't know.
therefore, you have to find threshold on the thing you DO know, then apply it on the thing you DONT know to evaluate your work.

[orlevit]

That is what I meant...
What are the thresholds that have been used in the test sets mentioned in the paper(those are fixed).

I didn't found any report on fixed value neither in the paper nor in the Github forum

[ltkhang]

That is what I meant...
What are the thresholds that have been used in the test sets mentioned in the paper(those are fixed).

I didn't found any report on fixed value neither in the paper nor in the Github forum

this is common sense on face recognition task, so they don't need to report it ...
Have a nice day bro, calm down and read more about other face recognition researches

[orlevit]

I really thank you for the answers, I appreciate the time you took for answering the questions.
I'll continue to read others researches as well :) cheers!

[alphinside]

Actually in this line https://github.com/deepinsight/insightface/blob/master/recognition/eval/verification.py#L172
the author try to find the best accuracy by grid searching threshold value between 0 and 4 with 0.01 diff point. So every best accuracy obtained at a specific dataset is possibly at different thresholds. If i'm not mistaken at LFW the best thresholds is around 1.4 you can try it yourself at different dataset. Hope this can help @orlevit

[orlevit]

Thanks! @alphinside

[ltkhang]

Actually in this line https://github.com/deepinsight/insightface/blob/master/recognition/eval/verification.py#L172
the author try to find the best accuracy by grid searching threshold value between 0 and 4 with 0.01 diff point. So every best accuracy obtained at a specific dataset is possibly at different thresholds. If i'm not mistaken at LFW the best thresholds is around 1.4 you can try it yourself at different dataset. Hope this can help @orlevit

I noticed this but it seems to be very "tricky".

I joined a contest, that I had to upload the inference code google colab with verification method and the fixed threshold value for contest holders. then they will apply to their test set and announce the result later.

In this case, how can I find my best threshold? I applied my inference model on multiple benchmark datasets and use avg value of threshold values, after that, I applied this avg threshold to re-verify on the above benchmark datasets, luckily the result still achieved SOTA. Finally, I decided to use avg threshold to submit.

[orlevit]

@ltkhang Could you please share the avg threshold that you used?