AI Program Struggles To Recognize Women’s Name

A recent investigation of an important AI Program reveals bias in name recognition.


Written by: Kelly Fisher


Del på TwitterDel på FacebookDel på TwitterDel på Twitter

What are the advantages of having AI or other computer-based programs to help society?  Often, we think about the types of mistakes that humans make that computers would avoid, including the biases that we as humans have.  But as we continue to see, that bias unfortunately is reflected when the people who are designing the programs unintentionally include some of their biases.  In a recent investigation of an open source dataset known as CoNLL-2003, which has been crucial in the development of machine learning and AI programs, it has become clear that something is missing.  Many women’s names. 

The data annotation firm Scale AI in a recent investigation found that there were many more men’s names listed in the dataset than women’s.  Male names were found to be mentioned 5 times more often than female ones.  It was also 5% more likely to miss a new woman’s name that was run through the program than a man’s name.   

One of the things that makes this problematic is the fact that CoNNL-2003 has been crucial in the development of other language programs which use AI.  In fact, the program has been cited 2,500 times in research literature, which makes the implications of these findings hard to pin down exactly.  What is clear though is that male bias is being perpetuated in computer programs which rely on CoNNL-2003. 

It is yet another important example of the ways in which we see how are own bias as humans unfortunately makes its way into the supposedly neutral and unbiased technologies we use.   

However, as the article concludes, there is a growing awareness among those who use these types of programs, and are developing them.  Often computer programming has been male dominated, but we do see more women entering the profession.  This is necessary if we as a global community are finally going to close the economic gender gap, and underscores the point that we need increased gender balance in more of our workforces.  Something that SHE Community is working towards.   

Read more about the investigation and its implications here.