We use cookies and other tracking technologies to improve your browsing experience on our site, analyze site traffic, and understand where our audience is coming from. To find out more, please read our privacy policy.

By choosing 'I Accept', you consent to our use of cookies and other tracking technologies.

We use cookies and other tracking technologies to improve your browsing experience on our site, analyze site traffic, and understand where our audience is coming from. To find out more, please read our privacy policy.

By choosing 'I Accept', you consent to our use of cookies and other tracking technologies. Less

We use cookies and other tracking technologies... More

Login or register
to apply for this job!

Login or register to start contributing with an article!

Login or register
to see more jobs from this company!

Login or register
to boost this post!

Show some love to the author of this blog by giving their post some rocket fuel 🚀.

Login or register to search for your ideal job!

Login or register to start working on this issue!

Engineers who find a new job through Golang Works average a 15% increase in salary 🚀

Blog hero image

Are facial recognition systems so bad?

Nikolay Donets 29 December, 2019 | 2 min read

Forbes posted an article 'London Police Facial Recognition ‘Fails 80% Of The Time And Must Stop Now’'. The question is, are facial recognition systems so bad in reality? To estimate it quantitively let's rephrase the question to the following one: what is the probability that an individual identified as a criminal is a real offender?

First, let's write a simple function in python to estimate it:

def if_prob(sensitivity: float, specificity: float, prob: float) -> float:
    """
    :param sensitivity: true positive results, probability
    :param specificity: true negative results, probability
    :param prob:        probability of the event
    """
    numerator = sensitivity * prob
    denominator = sensitivity * prob + (1.0 - specificity) * (1.0 - prob)
    return numerator / denominator

Now we are ready to answer the question. We can assume that the sensitivity of a given facial recognition system is 0.99 with a specificity of 0.99. Therefore we have true positive results in 99 trials out of 100, and we correctly identify false-positive 99 times out of 100. According to the "Annual Abstract of Statistics" (National Office of Statistics. 2009. Retrieved 23 January 2009), the prison population was about 149 people per 100,000 in 2007. From that, the probability of criminal might be estimated as 149/100000. With these numbers we have:

>>> if_prob(0.99, 0.99, 149/100_000)
0.1287150311512887

That is, a 12.8% probability to correctly identify a criminal. And to adjust probability to the claimed one in the Forbes article, let's set sensitivity equal to 99.5% and specificity to 99.5%:

>>> if_prob(0.995, 0.995, 149/100_000)
0.22896171487699016

That is, a 22.9% probability to correctly identify a real criminal, almost equal to what Forbes reported in the article. Given this, are there any ways to improve performance? One of the options might be a sequential identification with different conditions:

| Identifications in sequence | Probability of error         |
|-----------------------------|------------------------------|
| 1                           | (1-0.229)**1 = 0.771 = 77.1% |
| 3                           | (1-0.229)**3 = 0.458 = 45.8% |
| 5                           | (1-0.229)**5 = 0.272 = 27.2% |
| 7                           | (1-0.229)**5 = 0.161 = 16.1% |

To summarize this short note, remember to check business requirements to estimate the performance of systems and understand their limitations in real applications. Properties of a system might be outstanding, but it can produce miserable results in wrong conditions.

Originally published on donets.org

Author's avatar
Nikolay Donets
    Scala
    Python
    Machine Learning/Data Science
    NLP/NLU
    Google Cloud / AWS
    Apache Spark
    Recommendation Systems
    Personalization Systems
    Data Engineering
    Apache Flink
    Apache Airflow

Related Jobs

Related Issues

viebel / klipse-clj
viebel / klipse-clj
  • Open
  • 0
  • 0
  • Intermediate
  • Clojure
viebel / klipse
  • Open
  • 0
  • 0
  • Intermediate
  • Clojure
viebel / klipse
  • Open
  • 0
  • 0
  • Intermediate
  • Clojure
  • $100
viebel / klipse
  • 1
  • 0
  • Intermediate
  • Clojure
viebel / klipse
  • Open
  • 0
  • 0
  • Intermediate
  • Clojure
  • $80
viebel / klipse
  • Open
  • 0
  • 0
  • Advanced
  • Clojure
  • $80
viebel / klipse
  • Started
  • 0
  • 2
  • Advanced
  • Clojure
  • $180
viebel / klipse
  • Open
  • 0
  • 0
  • Intermediate
  • Clojure
viebel / klipse
  • Started
  • 0
  • 2
  • Intermediate
  • Clojure
  • $80

Get hired!

Sign up now and apply for roles at companies that interest you.

Engineers who find a new job through Golang Works average a 15% increase in salary.

Start with GithubStart with Stack OverflowStart with Email