Providing evidence that both personal and institutional biases can be reflected in software and artificial intelligence, a new study found that facial recognition technology has a much higher rate of failure when analyzing darker-skinned people, especially women.
M.I.T. Media Lab’s Joy Buolamwini published a paper entitled “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification” after studying the performance of three major facial recognition systems from Microsoft, IBM, and China’s Megvii. The tests sought to determine how well each system could identify the gender of people with varying skin tones.
A data set of 1,270 faces was used to compare the services, according to The New York Times, using elected officials from countries that have a high percentage of women in office. For diversity in race, the sources included three countries with predominantly darker-skinned populations, as well as three from Nordic countries where residents tend to be lighter skinned.
Buolamwini’s tests found that gender of lighter skinned men was identified correctly about 99 percent of the time. In contrast, the systems had a failure rate of 35 percent when attempting to identify the gender of darker-skinned females.
The study from M.I.T. is just the latest example showing that artificial intelligence can inherit bias from its human programmers, particularly against nonwhites. In 2016, the ACLU and NAACP called on the U.S. Department of Justice to investigate racial bias in law enforcement facial recognition , citing a study that found facial recognition algorithms were 5 to 10 percent less accurate with African Americans than caucasians.
“Such inaccuracies raise the risk that, absent appropriate safeguard, innocent African Americans may mistakenly be placed on a suspect list or investigated for a crime solely because a flawed algorithm failed to identify the correct suspect,” the NAACP and ACLU said in their letter to the DOJ.
The booming facial recognition market is expected to be driven in the near future by surveillance uses and government investment. One recent estimate forecast that facial recognition technology will bring in $6.84 billion per year by 2021.