Jul 8

Stanford Report: AI screening tools biased

Illustration shows a person using a magnifying glass to examine resumes, highlighting AI bias. Emphasizes selection of candidates with abstract figures.

For many job postings, AI screening tools recommend white candidates at higher rates than Black and Asian candidates, new research shows - Stanford Report. 

In a black-and-white scene, a man in a white tuxedo faces a uniformed officer in a tense interaction. Onlookers observe in the dimly lit room.

Rick: How can you close me up? On what grounds?


Captain Renault: I'm shocked, shocked to find that gambling is going on in here!

Well, color me 'not shocked' to see research confirm that AI tools are biased.

Stanford researchers analyzed more than 4 million job applications screened by the same AI vendor across 156 employers, mostly companies with $5 billion and up in annual revenue.

When they measured bias the way the law actually requires, job by job instead of averaged across the whole dataset, 26% of Black applicants and 15% of Asian applicants had applied to positions where the AI discriminated against their racial group.

If the algorithm had recommended them at the same rate as white applicants, 40,000 more applications would have advanced.And because so many employers use the same vendor, a candidate rejected at one company was statistically more likely to be rejected at the next.

The researchers called this "systemic rejection." I call it a digital redline.

As co-author Kathleen Creel put it: "As a single vendor comes to dominate decision-making in a space, their quirks or shortfalls can be present across that entire sector in a way that wasn't possible before.”

Ninety percent of U.S. employers use AI screening tools. Most rely on the same few vendors. This study only exists because the vendor shared its data, which almost never happens.This is almost certainly not the only story like this. It's just the one we can prove.

Employers are still legally responsible for the bias their tools produce. Asking whether your vendor self-certified is not enough. Audit job by job. Demand disaggregated data. That's where the harm hides. 

For disaggregated data analysis that actually checks for bias, I use Justice AI, the world's first decolonial AI assistant.

It audits documents, flags biased language, surfaces who is centered and who is erased, and maps the power structures hiding inside your data and policies. Built specifically for the people doing this work for real.

I use and recommend Justice AI. 

Courses