AI Hiring Algorithms: One Student's Fight for Answers

A Medical Student Takes On the Black Box of AI Hiring

A medical student armed with Python scripts and a deep sense of frustration spent 6 months investigating whether an AI-powered hiring algorithm silently killed his job applications before a human ever saw them. His story has ignited a fierce debate about transparency, fairness, and the growing power of automated screening tools that now stand between millions of candidates and their next career.

The unnamed student, whose account has circulated widely in tech and employment policy circles, described a familiar modern nightmare: dozens of carefully tailored applications sent out, zero interview callbacks. Rather than accept the silence, he decided to reverse-engineer the process — and what he found raises uncomfortable questions for every company using AI to filter talent.

Key Takeaways

• AI resume screeners now process an estimated 75% of applications at large companies before any human reviews them
• One medical student used Python to probe whether algorithmic filtering was responsible for his silence from employers
• His 6-month investigation revealed significant opacity in how candidates are scored and ranked
• Current U.S. federal law provides almost no transparency requirements for AI hiring decisions
• Companies like HireVue, Pymetrics (now part of Harver), and Workday dominate the automated screening market
• The EU's AI Act, set for full enforcement by 2026, classifies employment AI as 'high-risk' and mandates audits

The Quiet Rise of Algorithmic Gatekeepers

Automated hiring tools have exploded in adoption over the past 5 years. According to a 2023 survey by the Society for Human Resource Management, roughly 4 in 5 employers now use some form of automation in recruitment. For Fortune 500 companies, the figure is closer to 99%.

These systems promise efficiency. A single corporate job posting can attract 250 or more applicants, and AI tools claim to reduce time-to-hire by 50% or more. Products from vendors like Workday, ADP, and iCIMS use natural language processing to parse resumes, score candidates against job descriptions, and rank applicants — all in seconds.

But efficiency comes at a cost. Candidates have no visibility into how they are scored. They receive no explanation when they are filtered out. And unlike a human recruiter who might give a borderline candidate a chance, an algorithm draws hard lines.

The medical student's investigation highlighted exactly this problem. Despite strong qualifications, his applications appeared to vanish into a void. He suspected — and set out to prove — that the void had a name: an AI screening tool.

Python, Persistence, and Probing the Algorithm

The student's approach was methodical. Using Python and publicly available information about common applicant tracking systems (ATS), he created multiple versions of his resume with subtle variations. He altered keyword density, formatting, section ordering, and even file types.

His findings, while not peer-reviewed, echoed what researchers have documented for years:

• Resumes with specific keyword matches to job descriptions scored dramatically higher
• Formatting choices — such as using tables, headers, or non-standard fonts — caused some ATS platforms to misparse or entirely skip sections of his resume
• Gaps in employment history, even those explained in cover letters, appeared to trigger automatic downranking
• Submitting identical content in PDF versus Word format produced different parsing outcomes on at least 2 major platforms

His experiment was limited in scope. He couldn't access proprietary scoring models, and he acknowledged the difficulty of isolating algorithmic bias from simple market competition. But the exercise illuminated a critical truth: the system is opaque by design, and candidates are flying blind.

A Problem Bigger Than One Application

This story resonates because it is not unique. A 2024 report from Harvard Business School and Accenture found that automated hiring systems routinely filter out qualified candidates — an estimated 27 million 'hidden workers' in the U.S. alone who are capable of performing jobs but are screened out by rigid algorithmic criteria.

These tools often penalize candidates for:

• Career gaps (disproportionately affecting women and caregivers)
• Non-traditional educational backgrounds
• Job titles that don't exactly match the posting's language
• Lack of specific certifications even when equivalent experience exists
• Geographic location or commute distance estimates
• Overqualification, which some algorithms flag as a flight risk

Unlike a human recruiter who might recognize that a 'medical research coordinator' and a 'clinical research associate' involve overlapping skills, many AI systems treat these as distinct categories. The result is a system that optimizes for keyword matching rather than genuine competence.

Compared to older keyword-matching ATS platforms like Taleo (now part of Oracle), modern AI-driven tools claim to use more sophisticated NLP and even large language models. But critics argue that sophistication has not translated into fairness. It has simply made the black box harder to pry open.

The Legal and Regulatory Landscape Is Playing Catch-Up

New York City's Local Law 144, which took effect in July 2023, was the first major U.S. regulation to require bias audits of automated employment decision tools. Companies using AI to screen candidates in NYC must now commission annual third-party audits and publish summary results.

But the law has significant gaps. It does not require companies to explain individual decisions to candidates. It does not mandate disclosure of which AI tool is being used. And enforcement has been limited — the NYC Department of Consumer and Worker Protection issued only a handful of penalties in its first year.

At the federal level, the Equal Employment Opportunity Commission (EEOC) has signaled interest in regulating AI hiring tools under existing anti-discrimination law, but has not issued binding rules. The Biden administration's AI Executive Order from October 2023 mentioned employment AI but delegated specifics to agencies that have yet to act decisively.

Across the Atlantic, the EU AI Act takes a harder line. It classifies AI systems used in employment and worker management as 'high-risk,' requiring conformity assessments, transparency obligations, and human oversight. Full enforcement begins in 2026, and it could set a global standard — much as GDPR did for data privacy.

What This Means for Job Seekers, Employers, and Developers

For job seekers, the immediate lesson is tactical. Understanding how ATS and AI screening tools work is no longer optional — it is a survival skill in the modern job market. Services like Jobscan and Resume Worded have built entire businesses around helping candidates optimize resumes for algorithmic parsing.

But tactical adaptation only goes so far. The deeper issue is structural. When candidates must reverse-engineer opaque systems just to get their applications read, the labor market is not functioning efficiently. It is functioning for the convenience of employers and software vendors.

For employers, the risk is twofold. First, they are almost certainly missing qualified candidates. The Harvard/Accenture research makes this clear. Second, they face growing legal exposure as regulations tighten. Companies that cannot explain or audit their AI hiring processes will find themselves on the wrong side of compliance.

For AI developers building these tools, the student's story is a cautionary tale about the consequences of deploying systems without meaningful transparency. The industry needs to move beyond accuracy metrics and toward explainability. Candidates deserve to know why they were rejected — or at minimum, that an algorithm made the decision.

Looking Ahead: Transparency as the Battleground

The next 2 to 3 years will be pivotal for AI in hiring. Several forces are converging:

• State-level legislation is accelerating. Illinois, Maryland, and Colorado have all passed or proposed laws addressing AI in employment decisions
• The EU AI Act's enforcement timeline will pressure multinational companies to adopt global transparency standards
• Open-source auditing tools are emerging, giving researchers and advocates new ways to probe commercial hiring algorithms
• Public awareness is growing, fueled by stories like the medical student's investigation
• Major AI vendors are beginning to offer 'explainability dashboards,' though adoption remains voluntary

The fundamental tension is between efficiency and accountability. AI hiring tools deliver genuine value — processing thousands of applications quickly and consistently. But 'consistently' is not the same as 'fairly,' and speed means nothing if qualified candidates are being silently discarded.

One medical student with Python skills managed to scratch the surface of a system that affects millions. His investigation did not produce a smoking gun, but it produced something arguably more important: proof that the system resists scrutiny. And any system that resists scrutiny deserves more of it.

The question is no longer whether AI should be used in hiring. It is whether we will demand that it operates in the open — or continue to let algorithms make life-altering decisions in the dark.