Insightface: Verification or VR@FAR10^6 score on megaface

[yangfly]

How do you get Verification or VR@FAR10^6 score on megaface?
I run command like:

python run_experiment.py -s 1000000 \
    path_megaface_feature path_facescrub_feature _arcface.bin path_results/

And I got output with pretrained-R50E (cleaned)

Done matching! Score matrix size: 3530 1000000
Saving to /home/data/insight/megaface/raw_eval/r50/results_cm/otherFiles/facescrub_megaface_r50_112x112_1000000_1.bin
Computing test results with 1000000 images for set 1
Loaded 3530 probes spanning 80 classes
Loading from /home/data/insight/megaface/raw_eval/r50/results_cm/otherFiles/facescrub_facescrub_r50_112x112.bin
Probe score matrix size: 3530 3530
distractor score matrix size: 3530 1000000
Done loading. Time to compute some stats!
Finding top distractors!
Done sorting distractor scores
Making gallery!
Done Making Gallery!
Allocating ranks (1000080)
Rank 1: 0.965815

I carefully check all output log, and I couldn't find anything seems to be Verification score.
Is there any other command that can produce Verification or VR@FAR10^6 score?
@nttstar @bruinxiong or anyone who is willing to offer help.
Thank you.

• megaface devkit readme.txt

[nttstar]

Check the generated cmc_*.json file.

[yangfly]

Thank you, @nttstar !
And I have one more question.
In cmc_facescrub_megaface_r50_112x112_1000000_1.json
```cmc_1000000.json
{
"cmc" :
[
[ 0, 3, 128, 295648, 1000000 ],

            [
                    0.9658148884773254, 
                    0.9758517146110535,
                    0.9858820438385010,
                    0.9958863258361816,
                    1.0
            ]
    ],
    "roc" : 
    [

            [
                    0.0,
                    1.999999987845058e-08,
                    2.099999960591958e-07,
                    1.030000021273736e-06,
                    1.241999962076079e-05,
                    0.003779160091653466,
                    0.8626054525375366,
                    1.0
            ],

            [
                    0.9378262758255005,
                    0.9486702084541321,
                    0.9595271944999695,
                    0.9695510268211365,
                    0.9795748591423035,
                    0.9895856380462646,
                    0.9995964169502258,
                    1.0
            ]
    ],
    "traditional_cmc" : 
    [
            [ 0, 4, 269, 282990, 1000079 ],

            [
                    0.9668695926666260,
                    0.9772464036941528,
                    0.9872752428054810,
                    0.9972990155220032,
                    0.9999930858612061
            ]
    ]

}
`` I recognizeRank 1 score = 0.9658148884773254 But which one is the theVerificationorVR@FAR10^6score ? 0.9378262758255005or 0.9668695926666260` ?

[nttstar]

In 'roc' block, you can choose the corresponding value near FAR1e-6. The 4th line in your case is ok which is 0.9695510268211365

[yangfly]

Thank you!

[cavalleria]

my cmc_.json file
{
"cmc" :
[
[ 0, 275, 1000000 ],
[ 0.9857323169708252, 0.9957365989685059, 1.0 ]
],
"roc" :
[

    [
        0.0,
        9.999999939225290e-09,
        1.999999987845058e-08,
        5.999999785899490e-08,
        2.799999947455944e-07,
        0.0006921099848113954,
        1.0
    ],

    [
        0.8836455345153809,
        0.9026387333869934,
        0.9641355276107788,
        0.9759168028831482,
        0.9859926700592041,
        0.9960035085678101,
        1.0
    ]
],
"traditional_cmc" : 
[
    [ 0, 182, 1000079 ],
    [ 0.9860869646072388, 0.9961445331573486, 0.9999974966049194 ]
]

}
how can i get VR@FAR10^6(because no FAR1e-6 in roc block)?
thx

[cavalleria]

@nttstar

[fucker007]

在“ roc”块中，可以选择FAR1e-6附近的相应值。您的情况的第四行是可以的0.9695510268211365
Hello, sir, I dowload model zoon's model , it's named .3 LResNet34E-IR,ArcFace@ms1m-refine-v1, I follow the megaface test step , However ,I get a wrong result, can you help me.

{
       "cmc" :
       [

           [
               0,
               1,
               2,
              5,
              16,
              18,
              61,
             764,
              771,
              772,
              775,
              787,
              788,
              789,
              792,
              817,
              1075,
              5928,
              165791,
              1000000
          ],

          [
              0.2688922882080078,
              0.3066248297691345,
              0.3181847631931305,
              0.3284234106540680,
              0.4252053499221802,
              0.4379108846187592,
              0.4479672610759735,
              0.4579910635948181,
              0.5562961101531982,
              0.5738768577575684,
              0.5845385789871216,
            0.8930249810218811,
            0.9256544709205627,
             0.9413736462593079,
            0.9530377388000488,
            0.9631917476654053,
             0.9732025265693665,
             0.9832068681716919,
          0.9932111501693726,
              1.0
        ]
    ],