Scientists create AI that can suggest where to apply makeup to fool facial recognition

Scientists create artificial intelligence that can suggest where to apply makeup to a person’s face to fool facial recognition systems

  • Data scientists at Ben-Gurion University developed an AI software that tells you where to apply makeup to throw off facial recognition
  • Unlike other approaches, the AI was limited to natural hues to allow the person to blend in 
  • Participants wearing the ‘adversarial makeup’ recommended by the program  tricked the system 98.8 percent of the time
  •  Wearing the makeup lowered the FR success rate from 42.6 percent to 0.9 percent for women, compared to from 52.5 percent for 1.5 percent in men


You don’t have to wear a Halloween mask to avoid being detected by facial recognition software: a dab of makeup will do the trick, according to a new study.

Researchers at Ben-Gurion University in Beersheba, Israel, developed an artificial intelligence that shows users where to apply some foundation or rouge to fool face-recognition algorithms into thinking they’re looking at a different person.

The researchers tested their scheme against ArcFace, a machine-learning model that takes two facial images and determines the likelihood they’re the same person.

The team used 20 volunteers (10 men and 10 women) in a real-world environment using two cameras and a variety of lighting conditions and shooting angles.

Participants wearing ‘adversarial makeup,’ as recommended by the program, tricked the system 98.8 percent of the time.

Normally ArcFace correctly identifies a visage about half (47.6 percent) of the time, but that plummeted to 1.2 percent with the makeup on.

The researchers called that ‘below a reasonable threshold of a realistic operational environment.’

Scroll down for video

Computer scientists at Ben Gurion University developed an AI that could recommend subtle makeup applications to fool facial recognition system. The AI creaed a digital heat map of a participant’s face, highlighting areas that face recognition systems would likely peg as  identifiable.

Facial recognition programs look at what parts of a person’s face are most identifiable.

Data scientist Nitzan Guetta and her team attempted to mimic that approach, then determine how to subtly alter that unique feature without drawing too much attention.

The AI developed a digital heat map of a participant’s face, highlighting areas that face recognition systems would likely peg as highly identifiable.

A digital makeup projection then indicated where to apply makeup to alter the contours of the person’s face.

A participant wearing makeup recommended by the AI program (top) was able to fool a facial-recognition system. His makeover had an intensity score of just 1.81, compared to 2.30 when he applied random makeup

A participant wearing makeup recommended by the AI program (top) was able to fool a facial-recognition system. His makeover had an intensity score of just 1.81, compared to 2.30 when he applied random makeup

According to their study published on the open source server Arxiv, wearing makeup lowered the face recognition success rate from 42.6 percent to 0.9 percent for them, compared to 52.5 percent for 1.5 percent in men.

‘Facial contouring may become more than a TikTok trend, and such makeup may become the next wave to protect one’s privacy in public from automatic facial recognition systems,’ Mariann Hardey, a professor at Durham University’s business school, told New Scientist. 

While the participant is in initially recognized by the FR system (top), when they apply adversarial makeup (bottom), the system fails to pick them out.

While the participant is in initially recognized by the FR system (top), when they apply adversarial makeup (bottom), the system fails to pick them out.  

The intensity of the makeover was also recorded, and the average intensity score of the adversarial makeup was lower than random placement (2.64, compared to 2.85), ‘while being more effective at dodging the [facial recognition]’ the researchers said. 

The adversarial system was programmed to stick to natural hues when making recommendations.

There are other makeup techniques, like CV Dazzle, that use intentionally jarring patterns, contrasting colors and asymmetricity to throw off facial recognition technology.

But such outlandish makeup draws more attention to the wearer in the real world.

 

Advertisement