Facial Recognition

Facial Recognition Systems can be a double-edged sword in the current pandemic situation.  On the one hand, they are an excellent way to enhance security and enable touchless access to buildings. At the same time, face masks may impair the performance of facial recognition algorithms. This challenge can be overcome by upgrading the algorithms for face recognition with masks without compromising on accuracy. 

Challenges for Face Recognition Systems with Masks

Typically, facial recognition software works by scanning and grabbing data points on a person’s face such as nose structure, or gap between the eyes and compares it with an existing image in the database that is most likely without a mask. 

As more and more people begin to wear masks in public as a measure to prevent the spread of coronavirus, mobile phone owners immediately found a problem. With their mouths and noses covered, the phone’s facial recognition system that was used to unlock the device failed to function causing frustration to the users.

Before the Covid-19 pandemic hit, facial recognition software providers throughout the world were looking to install the technology everywhere, in schools, restaurants, casinos, airports etc. Face masks are posing a serious roadblock to the endeavors of a multi-million-dollar industry.                             

Due to the pandemic, facial recognition algorithms cannot be adequately tested for accuracy by organizations such as the National Institute of Standards and Technology which is one of the top authorities on face recognition accuracy ratings.

The other challenge for technology companies and security officials was whether the facial surveillance industry system would now be more susceptible to false matches and security breaches due to the detection of masked faces. With an entire face, there are a greater number of features for the AI algorithm to work on. With only eyes and eyebrows visible, there may be more similarities and false positives.

Workarounds Proposed by technological giants

Some technology companies claim that face recognition technology is not affected by masks and Artificial Intelligence can still identify and detect people with half-covered faces with reasonable accuracy.

A beta release of Apple iOS has updated its Face ID algorithm to account for people with masks. But facial recognition experts are still skeptical about the solution as they feel that facial recognition technology has proven to malfunction even without masks.  Studies say that the majority of face recognition algorithms have a false positive rate, especially for people of color.

A UK based company called Facewatch announced that its next algorithm release can detect and identify people based only on the eyes and eyebrow area. They claim that their technology has already been successful in recognizing people with glasses and hats. But they expect a slight delay in the identification of people with masks than without them.

SAFR, an AI-powered face recognition system, claims to have developed tools to identify faces with masks on. Their algorithms are being trained with images of individuals with masks. Several diverse sets of images of people with masks are fed to the algorithm to enable it to account for variations in age, gender, and race.

Several companies are leveraging digital enablement for enhancing their face recognition systems.  

NIST announced on May 1st that it would digitally add masks to the existing images of their photos database and run tests for accuracy of facial recognition. They will use software to digitally place synthetic masks on faces and test them for accuracy in detection. 

Amazon too is confident that its face recognition technology is robust enough to counter face masks.

Researchers of Wuhan University, China posted over 1000 pictures to GitHub curated from Instagram selfies. They then tagged the people in images who wore medical and non-medical masks to create solutions for face recognition problems due to COVID.  

With the dataset of the masked faces, developers can create face recognition and detection algorithms to identify travelers and others in masks who require authentication. Facial security scans can be implemented for pedestrians at checkpoints, and train stations. The dataset model for eye-focused face recognition with masks has been found to have an accuracy of over 95%. 

Other commercial companies such as Japan’s NEC that offer facial recognition services to border patrol and Customs at airports. The company vice-president confirms that their algorithms have already been tested with face masks during Asia’s flu season.

NEC uses another biometric for face detection that authenticates the person based on not only his face but also his iris. This method has a low rate of error and takes only a few seconds.

Sensory, a Silicon Valley AI firm has announced its face and voice combined biometric platform that can accurately detect users with masks as well as noisy surroundings. When faces are partially covered with masks, the voice biometric data is used for identification.


The strategy for face recognition systems at the times of pandemic must be to provide the end-user a safe, seamless, and contact-less experience with no compromise in accuracy. The facial recognition systems must be made adaptable, robust, and capable of working in a range of applications and venues.

About Author


Comments are closed