Posted on 05/19/2025 9:12:29 AM PDT by Red Badger
In a nutshell
AI resume screening tools showed strong racial and gender bias, with White-associated names preferred in 85.1% of tests and Black male names favored in 0% of comparisons against White males.
Bias increased when resumes were shorter, suggesting that when there’s less information, demographic signals like names carry even more weight.
Removing names isn’t enough to fix the problem, as subtle clues—like word choice or school name—can still reveal identity, allowing AI systems to continue filtering out diverse candidates.
=================================================================
SEATTLE — Every day, millions of Americans send their resumes into what feels like a digital black hole, wondering why they never hear back. Artificial intelligence is supposed to be the great equalizer when it comes to eliminating hiring bias. However, researchers from the University of Washington analyzing AI-powered resume screening found that having a Black-sounding name could torpedo your chances before you even make it to the interview stage.
A study presented at the AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society in October 2024 revealed just how deep this digital discrimination runs. The researchers tested three state-of-the-art AI models on over 500 resumes and job descriptions across nine different occupations. They found that resumes with White-associated names were preferred in a staggering 85.1% of cases, while those with female-associated names received preference in just 11.1% of tests.
The study found that Black male job seekers face the steepest disadvantage of all. In comparisons with every other demographic group—White men, White women, and Black women—resumes with Black male names were favored in exactly 0% of cases against White male names and only 14.8% against Black female names.
These aren’t obscure academic models gathering dust on university servers. The three systems tested—E5-mistral-7b-instruct, GritLM-7B, and SFR-Embedding-Mistral—were among the highest-performing open-source AI tools available for text analysis at the time of the study. Companies are already using similar technology to sift through the millions of resumes they receive annually, making this research particularly urgent for working Americans.
How the Bias Shows Up
These AI resume screening models convert resumes and job descriptions into numerical representations, then measure how closely they match using something called “cosine similarity,” essentially scoring how well a resume aligns with what the job posting is looking for.
Researchers augmented real resumes with 120 carefully selected names that linguistic studies have shown are strongly associated with specific racial and gender groups. Names like Kenya and Latisha for Black women, Jackson and Demetrius for Black men, May and Kristine for White women, and John and Spencer for White men.
When they ran more than three million comparisons between these name-augmented resumes and job descriptions, clear patterns emerged. White-associated names consistently scored higher similarity ratings, meaning they would be more likely to make it past initial AI screening to reach human recruiters.
Intersectional analysis, looking at how race and gender combine, revealed even more drastic disparities. Black men faced discrimination across virtually every occupation tested, from marketing managers to engineers to teachers. Meanwhile, the smallest gaps appeared between White men and White women, suggesting that racial bias often outweighs gender bias in these AI systems.
Critics might argue that removing names from resumes could solve this problem, but it’s not that simple. Real resumes contain numerous other signals of demographic identity, from university names and locations to word choices and even leadership roles in identity-based organizations.
Previous research has shown that women tend to use words like “cared” or “volunteered” more frequently in resumes, while men more often use terms like “repaired” or “competed.” AI systems can pick up on these subtle linguistic patterns, potentially perpetuating bias even without explicit demographic markers.
When researchers tested “title-only” resumes, containing just a name and job title, bias actually increased compared to full-length resumes. This suggests that in early-stage screening, where less information is available, demographic signals carry disproportionate weight.
An AI robot hiring manager shaking hands with a candidate
AI-powered resume screening is rapidly becoming the norm. According to industry estimates, 99% of Fortune 500 companies already use some form of AI assistance in hiring decisions. For job seekers in competitive markets, this means that algorithmic bias could determine whether their application ever reaches human eyes.
“The use of AI tools for hiring procedures is already widespread, and it’s proliferating faster than we can regulate it,” says lead author Kyra Wilson from the University of Washington, in a statement.
Unlike intentional discrimination by human recruiters, algorithmic bias operates at scale and often invisibly. A biased human might discriminate against a few candidates, but a biased AI system processes thousands of applications with the same skewed logic, amplifying its impact exponentially.
Can we fix AI bias in hiring?
Some companies are experimenting with bias mitigation techniques, such as removing demographic signals from resumes or adjusting algorithms to ensure more equitable outcomes. However, these approaches often face technical challenges and may not address the root causes of bias embedded in training data.
“Now that generative AI systems are widely available, almost anyone can use these models for critical tasks that affect their own and other people’s lives, such as hiring,” says study author Aylin Caliskan from the University of Washington. “Small companies could attempt to use these systems to make their hiring processes more efficient, for example, but it comes with great risks. The public needs to understand that these systems are biased.”
Current legal frameworks struggle to keep pace with algorithmic decision-making, leaving both job seekers and employers in uncharted territory. The researchers call for comprehensive auditing of resume screening systems, whether proprietary or open-source, arguing that transparency about how these systems work—and how they fail—is essential for identifying and addressing bias.
Of course, it’s important to remember that this research was presented in October 2024. While it’s still relatively new, LLMs are being updated quite often. Current versions of the systems tested may yield different results if they’ve since been updated.
In trying to remove human prejudice from hiring, we’ve accidentally created something worse: prejudice at machine speed. We’re letting AI make decisions about people’s livelihoods without adequate oversight. Until we acknowledge that algorithms inherit human prejudices, millions of qualified workers will keep losing out to systems that judge them by their names, not their abilities.
Paper Summary
Methodology
The researchers conducted an extensive audit of AI bias in resume screening using a document retrieval framework. They tested three high-performing Massive Text Embedding (MTE) models on 554 real resumes and 571 job descriptions spanning nine occupations. To measure bias, they augmented resumes with 120 carefully selected names associated with Black males, Black females, White males, and White females based on previous linguistic research. Using over three million comparisons, they calculated cosine similarity scores between resumes and job descriptions, then used statistical tests to determine if certain demographic groups were consistently favored. They also tested how factors like name frequency and resume length affected bias outcomes.
Results
The study found significant bias across all three AI models. White-associated names were preferred in 85.1% of tests, while Black names were favored in only 8.6% of cases. Male names were preferred over female names in 51.9% of tests, compared to female preference in just 11.1%. Intersectional analysis revealed Black males faced the greatest disadvantage, being preferred over White males in 0% of comparisons. The researchers validated three hypotheses about intersectionality and found that shorter resumes and varying name frequencies significantly impacted bias measurements.
Limitations The study relied on publicly available resume datasets that may not perfectly represent real-world job applications. Resumes were truncated for computational feasibility, potentially affecting results. The researchers used an external tool for occupation classification, which may be less accurate than manual coding. The study focused only on two racial groups (Black and White) and binary gender categories, limiting insights about other demographic groups. Additionally, the models tested were open-source versions that may differ from proprietary systems actually used by companies.
Funding and Disclosures
This research was supported by the U.S. National Institute of Standards and Technology (NIST) Grant 60NANB23D194. The authors note that the opinions and findings expressed are their own and do not necessarily reflect those of NIST. No competing interests or additional funding sources were disclosed in the paper.
Publication Information
This research was conducted by Kyra Wilson and Aylin Caliskan from the University of Washington in 2024. The paper “Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrieval” was presented in the Proceedings of the Seventh AAAI/ACM Conference on AI, Ethics, and Society (AIES 2024), 1578-1590. Association for the Advancement of Artificial Intelligence.
Courtney?
I once knew a (somewhat liberal)recruiter who told me that any resume containing some outrageous name went immediately to the circular file.
Called upon to make a serious, grown-up decision: What should I name my offspring? One that is public, permanent, and significant.
By "significant" I mean it signifies what the parent thinks about the child, the importance of a human life, how it can be influenced for the good, how it has meaning and worth, and how they feel about their responsibility as parents.
And of course, being fools, they fail, miserably, embarrassingly so.
And their offspring, who ought to be angry about it, are also unchurched, unread, unprincipled, and globally ignorant about everything.
Stupid names are fierce strong identifiers of stupidity, like tattoos, but are more easily removed.
85/15? The horror....
If blacks are 13% of the population, then they are still coming out ahead.
“Tushiaquandra, UARCO, LaDesmendia are probably is not in the AI database”
.
Got one for you.
La-a Limajelo.
Translation: LaDasha Limejello.
Saw it with my own eyes...
.
I’d never seen it. So funny! Thanks for clueing me in.
If an algorithm has a bias for names, you could assign a number and redact the names from the process.
But this is probably too obvious, and the only possible solution will be to have quotas. Right?
And what if an algorithm has a bias in favor of selecting the most qualified candidates, and this does not result in diversity utopia?
Who needs to see the Race block on applications when you only need to look at the name. Once again, black folks bring this on themselves. And not just males.
They are only content because they are too lazy to research or fact-check. The amount of false info/hallucinations is terrifying.
So is it possible A1 code can be written with built in biases? ... Color me surprised
I have read that AI ‘learns’ it’s stuff from information that it gathers from the Internet.
It has no idea if that information is real, bogus or just opinions of other people or even other AI’s. It just takes it all in and regurgitates it as factual.
It’s like a 9 year-old that is suddenly introduced to the Internet with no holds barred and no guardrails for its own protection. Everything the child reads is accepted as fact.
If the vast majority of the data the AI encounters on a certain subject is mired in racist rhetoric and biased opinions, then no wonder the answers it gives would be as well, only more so........................
With an otherwise equal resume (e.g., graduates from the same colleges with the same majors), whites and Asians are likely to have underlying SAT scores that are a few hundred points higher. That means that they are more likely to interview well and ultimately be chosen for positions (as long as DEI bias isn’t involved). Perhaps that’s the data driving these AI predictions of choices?
Eliminate the interview process altogether. Just let AI choose your employees.
You beat me to it. Sounds about right to me.
AI is racist. Who knew? 😆
I had a co-worker from Nigeria (naturalized American citizen with a DoD clearance). Very bright. A good team member. Her name was Nigerian and a huge challenge for the rest of the team to pronounce. I had multiple opportunities to speak with her on the phone and asked how to properly pronounce her name. I succeeded. In addition to speaking American English with no discernible accent, her native tongue is Igbo. I don't know what an AI would do with her resume, but it is a very good one.
I will share just the first names of my two co-workers with unusual names. The Nigerian co-worker is Chinenyem. The co-worker from Latvian heritage is Namejs. Both are top of the line co-workers.
the names below were taken from the rosters of college and pro football and basketball players, with some from local (L.A.) police blotters.
There are no imaginary names included.(Sorry - no punctuation)
Adalius
‘P Rae
Aaquan
Abrielle
Adewale
Ahmard
Aisha
Akisha
Aladrian
Alainian
Alando
Alaysia
Aldarius
Aleah
Aleeta
Alphe
Alterraun
Alvertis
Anqoinette
Anquan
Anterio
Antoine
Antrel
Antwaan
Antwan
Antwane
Aqeela
Arellious
Artell
Aundrae
Aurian
Aveion
Bakari
BenJarvus
Bijan
Bontae
Braylon
Briniyah
Camryn
Carlysia
Chanel
Chantelle
Chantrelle
Chantz
ChaToyya
Chaun
Chertisha
Chone
Chykie
Cleotus
Cletus
Contrell
Corinio
Correll
Craphonso
Crosetti
Da’Trayvan
Daequan
Dajohn
Dajuan
Damane
Damarcus
Damarlo
Damien
Danario
Danatus
Danieal
Dantrell
Danyell
Darcus
Daric
Darius
Darlanne
Darnell
Darrent
Darris
Dashawna
Dashon
Davontae
Davyeon
Dawan
Dayjia
DeAundre
Dechone
DeíMack
DeíMon
Deion
Deiondrick
Deione
DeíVion
DeíVon
Dejuan
Dekoda
Deliazard
DelJuan
Delmon
Deltha
Delwyn
DeMar
DeMarcus
Demario
Demeco
Demorrio
Denard
Denine
Denzel
Deon
Deonte
Dereona
Deron
Derricus
Desean
DeShaun
DeShawn
Deshea
Desmond
DeVarro
Devaughndre
DeVere
Devin
Devlin
DeZarey
Dezariah
DíAndre
DíAnthony
DíBrickashaw
Dionta
DiQuan
Diquone
DíQwell
DíVontrey
Domonique
Donrece
Dontarrius
Donteí
Dontral
Dontrell
Dontrelle
Dorissa
Dovante
Dravannti
Drayton
Du Juan
Dushawn
Dvon
Dwain
Dwyane
DyíOnne
Ean
Eboni
Edgerrin
Erasmus
Everettezz
Fabiene
Flozell
Fontel
Foswhitt
Frostee
Genarlow
Genean
Gerran
Gerris
Gervase
Gijon
Ha Lecia
Ikeam
Iofemi
Irdwin
Ishionte
Jabu
Jackeri
Jacquizz
Jahleighla
Jahvid
JaíCorey
JaíKuan
JaíShawn
JaJuan
Jakethiyah
Jamaal
Jamaine
Jamal
Jamar
Jamarcus
Jamario
Jamarius
Jameer
Jameisha
Jamiel
Jamiihr
Jamin
Jammal
Jamychal
Janarius
Janiyah
Janoris
Jarius
JaRon
Jarrad
Jarue
Jasmine
Jauan
Javarceay
Javares
Javarris
Javion
Jawann
Jaxeryus
Jaycen
Jayceon
Jayden
Jeh
Jehmu
Jemaiah
Jemalle
Jerame
Jerametrius
Jereme
Jerheme
Jermaine
Jermale
Jermareo
Jermichael
Jermil
Jermon
Jerramy
Jerrson
Jeshua
Jevon
Jheryl
JíMichael
Johnae
Johneisha
Johniya
Johnl
Joselio
Jrue
Jumaane
Junius
Juqua
Juwan
Jwon
Jyles
Jzamir
Kahlil
Kahyla
Kalima
Kalvin
Kalyn
Kambriel
Kamerion
Kanisha
Karington
Kashay
Kawann
Kayvon
Kedrin
Kee-ayre
Keion
Keishaun
Keiwan
Keiyanda
Kenisha
Kestahn
Keyaron
Keydrick
Keyon
Keyshard
Keyshawn
Keyunta
Khaled
Khalid
Khalif
Kheeston
Khreem
Kianna
Kiannah
Kieta
Kirlilah
Kiwaukee
Kjuana
Knowshon
Kwame
Kynika
LaDainian
Ladarious
LaíRoderick
LaíRoi
Lakeem
Lakeisha
Lakisha
Lamarcus
Lamarian
LaMarr
Laquan
Laquarry
LaRon
LaRoyce
LaSasha
LaShawn
Latarsha
LaTerra
LaTerryal
Latrell
LaTrisha
LaTroy
Lavelle
Laveranues
Lavernius
LeAndre
Leeshawn
LeíRon
Lemar
Lemaricus
LeMicah
Lendale
Lenyras
Leodis
Leomia
Letalvis
Levonne
Loletha
Lovell
Luster
Makayla
Marcedes
Marcellous
Marious
Markihe
MARKWAYNE
Marloe
Marquand
Marquel
Marquese
Marquest
Marquise
Marshawn
Martice
Martrez
Matterral
Maurkice
Meenakshi
Mewelde
Mijawon
Mikki
Mishawn
Mister
Montae
Montario
Montavious
Morkeith
Mossis
Mychal
Myhiesha
Mykal
Mykarsha
Nafloyd
Nakeisha
Naquan
Nischelle
Nurdeen
Nushawn
Nyasia
OíJahnae
OíJahnee
Omare
Omarius
Pharrell
Philonise
Pjai
Pleajhai
Purnell
Quan
Quanell
Quanis
Quinshon
Quintin
Quinton
Racquiha
Raheam
Raheem
RaíShon
Ramzee
Rashad
Rashard
Rasheed
Rashied
Rashieka
Ravon
Rayshaun
Rayshun
Renay
Reuele
Roisean
Rolandis
Rondarius
Ronnet
Ronyell
Roydell
Sakemeyia
Samardo
Santonio
Saquan
Sar-ron
Seandell
Seddrick
Senfronia
Shacoria
Shadayia
Shakira
Shamann
Shanesia
Shanetta
Shanice
Shanika
Shaniqua
Shaniya
Shantelle
Shantrelle
Shaquille
Shareece
Shawna
Shawnbrey
Shawne
Shawnte
Shelvin
Shemekia
Shequita
Sheriece
ShiíRenaja
Sinorice
SirVincent
Snorice
Starkiesha
Stephon
Stylez
Sycloria
Syesha
Ta-Nehisi
Ta’Kiyah
Tadarrius
Taiwan
Tamanika
Tamika
Tanard
Tarockus
Tarranisha
Tarvaris
Taurean
Taurisha
Tavin
Tayshaun
Terdell
Terrelle
Tervaris
Theandre
Thearon
Therrian
Toddrick
Toshmon
Tramon
Tranee
Travarous
Travon
Treavion
Tremaine
Trenard
Trequan
Tresonda
Treveyon
Trivel
Trumaine
Tyesha
TyíSheoma
Tyjana
Tyjuan
Tylan
Tyree
Tyreek
Tyreke
Tyshawn
Tywon
Unique
Uriel
Vanswan
Ventrell
Verron
Visanthe
Vondrell
Vonta
Vontae
Waynelle
Wondy
Xzavier
Yarisa
Zavarion
Zi’Aire
Zyair
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.