Kenya's AI Health Algorithm Penalizes the Poor

AI Healthcare Algorithm Drives Up Costs for Kenya's Poorest Citizens

An artificial intelligence system designed to predict how much Kenyans can afford to pay for healthcare has been systematically overcharging the country's poorest residents, according to a new investigation. The flawed algorithm, central to President William Ruto's flagship Social Health Authority (SHA) program launched in October 2024, was intended to deliver universal healthcare — but instead appears to be widening the gap between rich and poor.

The findings raise urgent questions about deploying AI in critical public services across developing nations, where data quality is often poor and algorithmic accountability remains virtually nonexistent. For the global AI industry, this case represents a cautionary tale about what happens when predictive models trained on incomplete or biased data are used to make life-or-death decisions about healthcare access.

Key Takeaways

• Kenya's AI-driven healthcare affordability algorithm overestimates what poor citizens can pay
• The Social Health Authority program replaced the decades-old National Hospital Insurance Fund (NHIF) in October 2024
• President Ruto promised universal healthcare amid widespread social unrest and protests
• The algorithm reportedly favors wealthier Kenyans, who receive more proportional cost assessments
• Millions of Kenyans in informal employment lack the financial data needed for accurate AI predictions
• The system was rolled out nationally without adequate testing or transparency about its methodology

How the Algorithm Gets Affordability Wrong

The SHA system uses an AI-powered model to assess each citizen's ability to pay for healthcare coverage. In theory, this approach should create a fairer system — those who earn more pay more, while the poorest receive subsidized or free access. In practice, the algorithm appears to be doing the opposite.

The core problem lies in data availability and quality. Kenya's economy is heavily informal, with an estimated 83% of the workforce operating outside formal employment structures, according to the Kenya National Bureau of Statistics. These workers — including street vendors, smallholder farmers, and domestic laborers — often lack bank accounts, tax records, or any digital financial footprint the algorithm can reliably interpret.

When the AI system encounters gaps in financial data, it appears to default to assumptions that overestimate earning capacity. Rather than flagging uncertainty and erring on the side of affordability, the model assigns cost tiers that many of Kenya's poorest families simply cannot meet. Meanwhile, wealthier citizens with well-documented incomes and formal employment records receive assessments that more accurately reflect their means — and in some cases, may even underestimate their capacity to contribute.

This pattern mirrors a well-documented phenomenon in algorithmic bias research. Studies from institutions like MIT and Stanford have repeatedly shown that AI systems trained on data from formal economies tend to perform poorly when applied to populations with different economic structures.

Ruto's Universal Healthcare Promise Unravels

President Ruto launched the SHA program amid intense political pressure. In mid-2024, Kenya experienced widespread Gen Z-led protests against proposed tax hikes and government spending. Ruto withdrew the controversial finance bill and pivoted to populist promises, including universal healthcare for all 54 million Kenyans.

The SHA was designed to replace the National Hospital Insurance Fund (NHIF), which had operated since 1966 but covered only a fraction of the population. Under the old system, Kenyans paid a flat rate based on income brackets — a simpler but arguably less equitable approach. The AI-driven replacement was marketed as a modern, data-driven solution that would ensure fairness.

However, the rapid rollout has been plagued by problems beyond the algorithm itself:

• Hospital rejections: Many healthcare facilities initially refused to accept SHA coverage due to delayed government reimbursements
• Registration failures: Millions of Kenyans struggled with the digital registration process, particularly in rural areas with limited internet
• Benefit gaps: Some treatments previously covered under NHIF were excluded from SHA coverage
• Transparency concerns: The government has not published details about the algorithm's methodology or the data it uses
• Legal challenges: Multiple court cases have been filed questioning the constitutionality of the mandatory transition

The result is a system that promised inclusion but has delivered confusion and, for the poorest citizens, higher costs than the program it replaced.

The Global Pattern of AI Bias in Public Services

Kenya's experience is far from unique. Governments worldwide have increasingly turned to AI systems to allocate public resources, often with similarly problematic results. The pattern is consistent enough that researchers have given it a name: automated inequality.

In the United States, algorithms used to determine eligibility for Medicaid and food assistance have been found to wrongly deny benefits to vulnerable populations. A 2019 study published in Science revealed that a healthcare algorithm used by major U.S. hospitals systematically underestimated the health needs of Black patients compared to white patients with similar conditions. The algorithm used healthcare spending as a proxy for health needs — but because Black patients historically had less access to healthcare, they spent less, and the AI interpreted this as being healthier.

Similar issues have emerged in the Netherlands, where an AI-powered fraud detection system used by the tax authority wrongly flagged thousands of families — disproportionately those with immigrant backgrounds — as childcare benefit cheats. The scandal ultimately brought down the Dutch government in 2021.

What makes Kenya's case particularly concerning is the lack of institutional safeguards. Unlike the U.S. or Europe, Kenya does not have comprehensive data protection enforcement, algorithmic accountability frameworks, or well-funded civil society organizations capable of auditing government AI systems at scale. The Kenya Data Protection Act of 2019 exists on paper but enforcement remains limited.

What This Means for AI Deployment in Developing Nations

The Kenya healthcare case highlights a critical challenge for the global AI industry: the tools and models developed primarily in wealthy, data-rich economies often fail when deployed in fundamentally different contexts. This has significant implications for several stakeholders.

For AI developers and vendors, the lesson is clear — algorithmic systems must be designed with data scarcity in mind. Models that perform well with complete financial records may produce harmful outcomes when applied to populations where such data does not exist. Robust uncertainty quantification and conservative default assumptions are essential, not optional.

For governments in developing nations, the case underscores the risks of adopting AI solutions without adequate technical capacity to evaluate, monitor, and correct them. The appeal of 'leapfrogging' legacy systems with cutting-edge technology is understandable, but the consequences of getting it wrong fall disproportionately on those least able to advocate for themselves.

For international organizations and donors who often fund such digital transformation projects, the Kenya example demands greater scrutiny of AI procurement processes. Key questions include:

• Who built the algorithm, and what data was it trained on?
• Was the system independently audited before deployment?
• Are there mechanisms for citizens to challenge algorithmic decisions?
• Is there ongoing monitoring for disparate impact across income groups?
• What fallback systems exist if the AI produces harmful outcomes?

The Broader AI Ethics Reckoning

This investigation arrives at a moment when the global AI industry is grappling with questions of responsible deployment more broadly. Companies like OpenAI, Google DeepMind, and Anthropic have invested heavily in AI safety research, but their focus has primarily been on frontier model risks — existential threats, misinformation, and autonomous weapons — rather than the mundane but equally damaging harms caused by predictive algorithms in public services.

The gap between AI safety discourse in Silicon Valley and the lived reality of algorithmic harm in Nairobi is stark. While Western tech leaders debate hypothetical superintelligence scenarios, millions of Kenyans are experiencing the concrete consequences of a biased algorithm right now.

Organizations like the Algorithmic Justice League, founded by Joy Buolamwini, and the Distributed AI Research Institute (DAIR), founded by Timnit Gebru, have long argued that AI harms disproportionately affect marginalized communities. Kenya's healthcare debacle provides further evidence for their position.

Compared to India's Aadhaar-linked welfare system, which faced similar criticisms about excluding the poorest from benefits, Kenya's SHA rollout appears to have learned few lessons from earlier implementations elsewhere. India's experience, documented extensively by researchers like Reetika Khera, showed that biometric and algorithmic systems for welfare distribution consistently failed the most vulnerable — those without stable addresses, consistent biometric data, or digital literacy.

Looking Ahead: Can Kenya Course-Correct?

The path forward for Kenya's healthcare system remains uncertain. Several outcomes are possible in the coming months.

First, the Kenyan government could commission an independent audit of the SHA algorithm, bringing in external experts to evaluate its methodology, training data, and outcomes across income groups. This would be the most constructive response, but it requires political will that may be lacking given the program's status as a presidential flagship initiative.

Second, civil society organizations and legal advocates could force changes through the courts. Several cases are already pending, and a judicial finding that the algorithm discriminates against the poor could compel the government to revise or replace the system.

Third, the government could adopt a hybrid approach, maintaining the AI system for formal-sector workers with reliable financial data while using simpler, community-based assessment methods for informal workers. This would sacrifice technological elegance for practical fairness.

What remains clear is that deploying AI in critical public services without transparency, accountability, and robust testing is a recipe for harm — particularly in contexts where the data infrastructure cannot support the algorithmic ambitions. Kenya's experience should serve as a warning to every government considering similar digital transformations: the promise of AI-driven efficiency means nothing if it comes at the expense of the people the system was built to serve.

The global AI community — developers, policymakers, and civil society alike — must reckon with a fundamental truth: algorithms are not neutral. They encode the biases of their creators and the gaps in their training data. When those biases determine who can access healthcare, the stakes could not be higher.