Facenet: Compare.py gives inaccurate result

[MrXu]

I have a simple testing to compare openface and facenet.
For two photos of same person, facenet compare.py gives 0.295 as distance while openface gives 0.029 as distance.
For another two photos of two different people, facenet compare.py gives 0.60 as distance while openface fives 0.83.
From the LFW result, facenet should have better accuracy than openface's nn1.small model.
May I know what may cause this inaccuracy?

[davidsandberg]

Hi @MrXu,
I haven't looked in to this alot but it sounds like what you are actually seeing is the curse of dimensionality.
So my guess is that for the openface embeddings (of size 128) the average distance is further apart but they are also noisier compared to the facenet embeddings (of size 1792). And that's why the facenet embeddings can produce better results when compared in terms of P(distance>Threshold).

From the wikipedia-page:
A further development of this phenomenon is as follows. Any fixed distribution on ℝ induces a product distribution on points in ℝd. For any fixed n, it turns out that the minimum and the maximum distance between a random reference point Q and a list of n random data points P1,...,Pn become indiscernible compared to the minimum distance:[5]
...
This is often cited as distance functions losing their usefulness (for the nearest-neighbor criterion in feature-comparison algorithms, for example) in high dimensions. However, recent research has shown this to only hold in the artificial scenario when the one-dimensional distributions ℝ are independent and identically distributed.[6] When attributes are correlated, data can become easier and provide higher distance contrast and the signal-to-noise ratio was found to play an important role, thus feature selection should be used.[6]

[MrXu]

Hi @davidsandberg , thanks for your reply. The higher dimension should be the cause.
So, how should we determine the threshold and compute the possibility of the two faces being the same person?

[davidsandberg]

I guess it depends on the application, but for example when evaluating on LFW (face verification task) the threshold is decided using cross validation.