How Algorithm Bias Affects Hiring Choices, Favouring Men Over Women: Study

To the vast majority of us, artificial intelligence is a complex, obscure world. Many institutions make decisions based on artificial intelligence (AI) systems using machine learning (ML), whereby a series of algorithms takes and learns from massive amounts of data to find patterns and make predictions. As per new research, algorithm bias is also responsible for exhibiting and amplifying gender inequality.

The work, which appears in the journal Proceedings of the National Academy of Sciences (PNAS), is among the latest to uncover how artificial intelligence (AI) can alter our perceptions and actions.

“There is increasing concern that algorithms used by modern AI systems produce discriminatory outputs, presumably because they are trained on data in which societal biases are embedded,” says Madalina Vlasceanu, a postdoctoral fellow at the New York University’s Department of Psychology and the paper’s lead author. “As a consequence, their use by humans may result in the propagation, rather than reduction, of existing disparities.”

Technology experts have expressed concern that algorithms used by modern AI systems produce discriminatory outputs, presumably because they are trained on data in which societal biases are ingrained. In that case, they absorb sexism, casteism, racism and every disparity, further producing results that are skewed and can be co-opted for oppressive uses.

Image from Unsplash

The researchers conducted multiple studies to assess a) whether the bias in algorithmic output correlates with social fissures and b) if so, what impact it has on people when the algorithm reinforces the biases they have been conditioned with. The first research performed through Google image show searches for the gender-neutral term "person" in different countries in two analyses--one involving 37 countries and another involving 52 countries--in the countries' dominant language and found that in countries with higher gender inequality, the first 100 search results contained more photos of men than women.

In a further experiment, when participants in the United States were shown Google image search results for unfamiliar job titles that included more men than women, they were more likely to state that they would hire a man to do the job, suggesting that gender inequality is mirrored in internet search algorithms and can lead to biased decision-making, according to the authors.

"Certain 1950s ideas about gender are actually still embedded in our database systems,” said Meredith Broussard, author of Artificial Unintelligence: How Computers Misunderstand the World and a professor at NYU’s Arthur L. Carter Journalism Institute, earlier this year.

AI systems based on incomplete or biased data can lead to inaccurate outcomes that infringe on people’s fundamental rights and will have a profound impact on women’s short and long-term psychological, economic and health security. It can also reinforce and amplify existing harmful gender stereotypes and prejudices.

The design and use of artificial intelligence models in different industries can significantly disadvantage women’s lives. And while there is agreement that lots of good data can indeed help close gender gaps, there remain concerns that if the “right” questions are being asked in the data collection process.

Suggested Reading: Do Social Media Algorithms Erode Our Ability To Make Decisions Freely?