Face masks are more powerful than we once thought. While we already knew they could help curb the spread of COVID-19, it seems they also have the power to break facial recognition algorithms.
Researchers at the US National Institute of Standards and Technology (NIST) have recently found that the now widely used face mask causes the error rate of commonly used facial recognition algorithms to rise. The study found that the error rate spiked to between 5% and 50%.
“Even the best of the 89 commercial facial recognition algorithms tested had error rates between 5% and 50% in matching digitally applied face masks with photos of the same person without a mask,” they said.
NIST explored how the algorithms were able to perform “one-to-one” matching, by comparing a photo of a person to a different photo of the same person. They digitally applied mask shapes (of varying sizes, shapes, colours, and face coverage) to the original photos and tested the performance of the algorithm.
“We can draw a few broad conclusions from the results, but there are caveats,” said Mei Ngan, a NIST computer scientist and an author of the report. “None of these algorithms were designed to handle face masks, and the masks we used are digital creations, not the real thing.”
Keeping the limitations in mind, the report tells us a few things. First, algorithm accuracy with masked faces declined substantially across the board. Second, masked images more frequently caused algorithms to be unable to process a face. Third, the more of the nose a mask covers, the lower the algorithm’s accuracy. Lastly, the shape and colour of the mask matters.
For additional information about the report, the team has posted details online, and the full report, Ongoing Face Recognition Vendor Test (FRVT) Part 6A: Face recognition accuracy with face masks using pre-COVID-19 algorithms, offers details of the performance of each algorithm.
“With respect to accuracy with face masks, we expect the technology to continue to improve,” said Ngan. “But the data we’ve taken so far underscores one of the ideas common to previous FRVT tests: Individual algorithms perform differently. Users should get to know the algorithm they are using thoroughly and test its performance in their own work environment.”
“With the arrival of the pandemic, we need to understand how face recognition technology deals with masked faces,” she said. “We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks. Later this summer, we plan to test the accuracy of algorithms that were intentionally developed with masked faces in mind.”