What Could Go Wrong With Trump's AI Safety Tests

Trump Reverses Course on AI Safety Testing

The Trump administration has effectively acknowledged that the Biden-era approach to AI safety testing had merit — a stunning reversal that raises as many questions as it answers. After dismantling President Biden's October 2023 executive order on AI safety within days of taking office, the current administration is now quietly building its own framework for evaluating AI systems before deployment, one that experts say bears striking resemblance to the policies it scrapped.

But while the policy U-turn itself made headlines, AI researchers, policy analysts, and industry insiders are far more concerned about what happens next. According to multiple experts, Trump's version of AI safety testing could fail in ways that leave the public more vulnerable than having no testing at all — creating a false sense of security while lacking the rigor, independence, and enforcement mechanisms needed to genuinely protect against catastrophic AI risks.

Key Takeaways at a Glance

• Policy reversal: Trump's administration is reintroducing AI safety testing after dismantling Biden's framework in early 2025
• Expert skepticism: Researchers warn the new approach may prioritize industry self-regulation over independent oversight
• Enforcement gaps: Unlike Biden's executive order, the new framework reportedly lacks clear penalties for non-compliance
• Voluntary vs. mandatory: The testing regime may rely on voluntary participation from AI companies, echoing failed self-regulation models
• Timeline concerns: Rapid AI development means any testing framework could be outdated before implementation
• Political influence: Critics warn that close ties between the administration and major AI companies could compromise testing integrity

Why Biden's Original Framework Was Scrapped

When Trump took office in January 2025, one of his earliest actions was revoking Executive Order 14110, Biden's sweeping directive requiring AI companies to share safety test results with the federal government before releasing powerful models. The administration argued at the time that the regulations stifled innovation and placed unnecessary burdens on American AI companies competing against Chinese rivals like DeepSeek and Baidu.

The original Biden framework required companies developing AI models above certain computational thresholds — specifically those trained using more than 10^26 floating-point operations — to submit red-team testing results to the government. It also established the AI Safety Institute within the National Institute of Standards and Technology (NIST) to develop evaluation benchmarks.

Industry lobbying played a significant role in the rollback. Companies including OpenAI, Google DeepMind, Anthropic, and Meta had expressed mixed feelings about the requirements, with some executives privately arguing that compliance costs could reach tens of millions of dollars annually. The political calculus was simple: remove barriers, accelerate development, maintain America's lead.

But within months, a series of high-profile AI incidents — from deepfake-driven financial fraud schemes to autonomous agent systems behaving unpredictably — forced the administration to reconsider.

The New Framework's Fundamental Flaws

Experts have identified several critical vulnerabilities in the Trump administration's emerging approach to AI safety testing. Unlike Biden's framework, which leveraged existing federal infrastructure and established clear reporting requirements, the new system reportedly relies heavily on industry self-assessment and voluntary compliance.

'The fundamental problem is that you're asking the fox to guard the henhouse,' said Dr. Yoshua Bengio, a Turing Award-winning AI researcher who has been vocal about existential AI risks. Several other prominent researchers have echoed this concern.

The key structural problems include:

• No independent verification: Companies would largely conduct their own safety evaluations without mandatory third-party audits
• Undefined benchmarks: The framework reportedly lacks specific, measurable criteria for what constitutes a 'safe' AI system
• Regulatory capture risk: Former AI industry executives now serving in government roles could influence which tests are required
• Inadequate scope: The testing requirements may not cover open-source models, military AI applications, or systems deployed by smaller companies
• Funding shortfalls: The AI Safety Institute has seen its budget reduced by approximately 40% compared to Biden-era levels
• No whistleblower protections: Employees who identify safety failures have limited legal protections for reporting concerns

The Self-Regulation Trap Has Failed Before

History offers a sobering precedent for voluntary industry safety testing. The Boeing 737 MAX disasters of 2018 and 2019, which killed 346 people, were partly attributed to the FAA's decision to delegate safety certification to Boeing itself. The parallels to AI safety testing are uncomfortable but instructive.

In the social media space, companies like Meta (then Facebook) spent years promising to self-regulate content moderation and user safety. Independent research consistently showed that internal safety mechanisms were insufficient, leading to documented harms ranging from teen mental health crises to the facilitation of genocide in Myanmar.

AI systems present even greater challenges for self-regulation. Unlike aircraft or social media platforms, frontier AI models exhibit emergent behaviors — capabilities that appear unexpectedly as models scale up. OpenAI's GPT-4 demonstrated reasoning abilities that were not explicitly trained, and Anthropic's Claude has shown unexpected patterns in safety evaluations. Testing for behaviors you cannot predict requires fundamentally different approaches than traditional software quality assurance.

'You cannot test for capabilities you don't know exist,' noted Dr. Stuart Russell, a professor of computer science at UC Berkeley and author of 'Human Compatible.' 'Any safety framework that doesn't account for emergence is essentially testing yesterday's risks while tomorrow's risks go unexamined.'

Political Entanglements Threaten Testing Independence

Perhaps the most alarming concern raised by experts involves the political dynamics surrounding AI safety under the current administration. Several major AI company executives have cultivated close relationships with the Trump White House, raising questions about whether safety testing would be applied evenhandedly.

Elon Musk's dual role as a government advisor through the Department of Government Efficiency (DOGE) and as the owner of xAI — which develops the Grok family of AI models — represents an unprecedented conflict of interest. Any safety testing framework that Musk influences could theoretically be designed to favor his own company's approach while disadvantaging competitors.

Similarly, Sam Altman of OpenAI reportedly donated $1 million to Trump's inaugural fund, and Sundar Pichai of Google has maintained regular communication with administration officials. These relationships create an environment where safety standards could be shaped by commercial interests rather than public welfare.

Compared to the European Union's AI Act, which establishes an independent regulatory body with enforcement powers and fines up to 7% of global revenue for violations, the Trump administration's approach lacks institutional independence. The EU framework, which began enforcement in February 2025, has already prompted companies to modify their deployment practices in European markets.

What This Means for Developers and Businesses

For AI developers and businesses operating in the United States, the uncertainty surrounding safety testing creates a challenging compliance landscape. Companies face several practical dilemmas.

First, planning for regulatory whiplash is now a business necessity. Organizations that dismantled Biden-era compliance programs must now rebuild safety testing infrastructure, potentially at greater cost than maintaining the original systems would have required. Estimates from the Information Technology Industry Council suggest that mid-size AI companies spent between $500,000 and $2 million adapting to the initial rollback.

Second, the lack of clear standards means companies must make judgment calls about what level of safety testing is sufficient. This creates liability risks — if an AI system causes harm and the company's testing is later deemed inadequate, the absence of clear government standards could actually increase legal exposure rather than reduce it.

Third, international fragmentation is accelerating. Companies operating globally must now navigate the EU AI Act, the UK's AI Safety Institute requirements, China's algorithm registration system, and whatever framework the US ultimately adopts. This patchwork of regulations favors large companies with dedicated compliance teams over startups and smaller developers.

Looking Ahead: The Clock Is Ticking

The stakes of getting AI safety testing right are escalating rapidly. Frontier AI models are advancing at an unprecedented pace, with several companies expected to release systems capable of autonomous scientific research, complex financial trading, and advanced code generation within the next 12 to 18 months.

The Trump administration reportedly plans to finalize its AI safety framework by late Q3 2025, but experts warn this timeline may be too slow. By the time regulations are in place, the AI systems they were designed to evaluate may already be a generation behind the cutting edge.

Several concrete steps could strengthen the framework, according to policy experts:

• Mandate independent third-party safety audits for all frontier AI models
• Restore full funding to the NIST AI Safety Institute
• Establish clear computational thresholds that trigger mandatory testing
• Create whistleblower protections for AI safety researchers
• Develop international reciprocity agreements with EU and UK regulators
• Require public disclosure of safety evaluation results

The irony of the situation is hard to miss. The administration that dismantled Biden's AI safety infrastructure now faces the challenge of rebuilding it — likely at greater cost, with less institutional knowledge, and under more intense time pressure. Whether the result will genuinely protect the public or merely provide political cover remains the critical question that experts say keeps them up at night.

As AI systems grow more capable and more deeply embedded in critical infrastructure, the margin for error in safety testing shrinks to zero. Getting this wrong is not just a policy failure — it could be a catastrophe with consequences that no executive order, however well-intentioned, can reverse.