🧪

NIST FRVT Results Analysis

Summary:

  • Trueface is ranked first in the world for template comparison speed (as of 01/12/2022) in NIST FRVT and is capable of comparing two biometric templates in 103 nanoseconds using only a single thread. Using multiple threads and additional compression techniques, Trueface can achieve speeds of up to 8.73 nanoseconds per comparison.
  • Trueface achieved the third-highest accuracy ranking by a western firm in the NIST FRVT 1 to N Identification, Visa Kiosk category.
  • Trueface can correctly identify 99.04% of identities at a false positive rate of less than 0.000001.
    • FRR = 0.000001, FAR = 0.0096
  • Trueface can correctly identify 99.52% of identities at a false positive rate of less than 0.00001.
    • FRR = 0.00001, FAR = 0.0048

The National Institute of Standards and Technology (NIST) is a US government agency that evaluates advanced technologies for the benefit of the public market. The Facial Recognition Vendor Test (FRVT) is considered the most rigorous and difficult evaluation for facial recognition technology. You can find more information regarding the evaluation here.

Trueface has made four submissions to the FRVT 1 to 1 competition, and one submission to the FRVT 1 to N competition. The following is a summary of the results.

FRVT 1 to 1

The following box plots show the distribution of False Negative Match Rates (FNMR) at the specified False Match Rates (FMR) for the 301 listed algorithms. FNMR (or false negative rate) is the proportion of mated comparisons producing a score below the threshold set to achieve the specified FMR. FMR (or false positive rate) is the proportion of impostor comparisons producing a score at or above that threshold. A lower FNMR indicates a more accurate algorithm.

The magenta line represents Trueface's third submission and uses the TFV5 model. As can be seen, Trueface has made consistent incremental improvements with each submission in nearly all categories.

image

Template comparison time is measured as the amount of time taken to generate a similarity metric given two biometric templates. A lower template comparison time allows for comparing more templates in less time, which is immensely important when performing 1 to N searches on large collections containing millions of biometric templates. For more information on why this is important, please refer to this blog post. As of 01/12/2022, Trueface is ranked first in the world for template comparison speed.

image

FRVT 1 to N

The following box plots show the distribution of False Negative Identification Rates (FNIR) for a False Positive Identification Rate (FPIR) of 0.003 in Identification Mode. FNIR is the proportion of mated searches failing to return the mate above the threshold. FPIR is the proportion of non-mated searches producing one or more candidates above the threshold. A lower FNIR indicates a more accurate algorithm.

The magenta line represents Trueface's first submission and uses the TFV5 model. Out of 262 entries, Trueface achieved the third-highest ranking by a western firm in the Visa Kiosk category.

image

image

Additional Resources:

🧪
Trueface NIST FRVT 1 to 1, Submission 2 Analysis
🧪
Trueface NIST FRVT 1 to 1, Submission 1 Analysis

Disclaimer: Results shown from NIST do not constitute an endorsement of any particular system, product, service, or company by NIST. For more information, visit https://www.nist.gov/programs-projects/face-recognition-vendor-test-frvt-ongoing

image