US to Safety Test New AI Models From Google, Microsoft, xAI

The U.S. Commerce Department has secured new agreements with Google, Microsoft, and Elon Musk's xAI to conduct safety testing on their latest artificial intelligence models before and after public release. The deals mark a significant continuation — and expansion — of AI safety commitments that originated under the Biden administration, signaling that government oversight of frontier AI development retains bipartisan momentum regardless of the political landscape in Washington.

The agreements formalize a process in which the U.S. AI Safety Institute (AISI), housed within the National Institute of Standards and Technology (NIST), gains pre-deployment and post-deployment access to new AI systems for evaluation. This represents one of the most concrete steps the federal government has taken to embed itself in the AI development pipeline.

Key Takeaways at a Glance

• Google, Microsoft, and xAI have signed voluntary agreements with the Commerce Department for government safety testing of new AI models
• The pacts build on Biden-era memorandums of understanding originally signed by companies including OpenAI and Anthropic
• The U.S. AI Safety Institute will conduct evaluations both before and after models are released to the public
• xAI's inclusion is notable given Elon Musk's close advisory relationship with the Trump administration
• The agreements are voluntary, not legally binding mandates, but carry significant reputational weight
• Testing will focus on areas including national security risks, biosecurity, cybersecurity, and societal impacts

Biden-Era Foundations Get a Trump-Era Refresh

The original AI safety commitments trace back to July 2023, when the Biden White House brokered voluntary pledges from 15 leading AI companies. Those initial agreements included commitments to share safety test results with the government, invest in cybersecurity, and research societal risks.

The AI Safety Institute was subsequently established in late 2023 within NIST to serve as the federal government's technical hub for AI evaluation. Under Biden, AISI signed memorandums of understanding with OpenAI and Anthropic to test their frontier models.

What makes the latest round of agreements noteworthy is the political context. The Trump administration initially signaled skepticism toward AI regulation, revoking Biden's sweeping AI executive order in January 2025 and emphasizing a more industry-friendly approach. Yet the continuation of these safety testing pacts suggests a pragmatic recognition that some guardrails remain necessary — particularly as AI models grow more capable and potentially more dangerous.

The Commerce Department under Secretary Howard Lutnick appears to be threading a careful needle: maintaining safety oversight without imposing the kind of heavy-handed regulation that Silicon Valley has lobbied against.

xAI's Inclusion Raises Eyebrows and Questions

Perhaps the most striking element of the new agreements is the inclusion of xAI, Elon Musk's AI startup. Musk serves as a senior adviser to President Trump and leads the Department of Government Efficiency (DOGE), giving him extraordinary influence over federal policy.

xAI's flagship model, Grok, has faced criticism from AI safety researchers for having fewer content guardrails than competing systems from OpenAI or Anthropic. The company has moved aggressively to scale its infrastructure, building a massive supercomputer cluster in Memphis, Tennessee, powered by 100,000 Nvidia GPUs.

The decision to submit to government safety testing could serve multiple purposes for xAI:

• Legitimacy: Voluntary compliance signals maturity and responsibility to enterprise customers
• Market access: Government testing could open doors to lucrative federal contracts
• Political cover: Participation provides a defense against future criticism of Musk's dual role as AI developer and government adviser
• Competitive positioning: Being tested alongside Google and Microsoft places xAI in the same tier as established players

Critics, however, question whether meaningful oversight is possible when the subject of testing has such close ties to the administration conducting the tests. Transparency around the process will be critical to maintaining public trust.

What the Safety Testing Actually Involves

The AI Safety Institute employs a multifaceted evaluation framework that goes well beyond simple benchmark testing. AISI's approach encompasses several critical risk categories that reflect the government's evolving understanding of how advanced AI could cause harm.

Testing protocols typically include red-teaming exercises, in which specialized teams attempt to elicit dangerous outputs from models. These exercises probe whether AI systems can be manipulated to provide instructions for creating biological or chemical weapons, generate sophisticated cyberattack code, or produce convincing disinformation at scale.

The evaluations also assess more subtle risks:

• Model autonomy: Can the AI take actions beyond its intended scope?
• Deception capability: Does the model demonstrate the ability to mislead users or evaluators?
• Dual-use potential: Could benign capabilities be repurposed for harmful applications?
• Bias and discrimination: Does the model exhibit systematic unfairness across demographic groups?
• Robustness: How does the model behave under adversarial conditions or edge cases?

Unlike the EU AI Act, which imposes legally binding requirements and fines of up to 7% of global revenue for non-compliance, the U.S. approach remains voluntary. Companies agree to share model access and technical documentation, but there are no statutory penalties for failing to address issues the Safety Institute identifies. This stands in stark contrast to the European regulatory framework, which took effect in stages throughout 2024 and 2025.

Industry Context: A Crowded and Competitive Safety Landscape

The new agreements arrive at a pivotal moment in the AI industry. Frontier model development has accelerated dramatically, with Google's Gemini 2.5, OpenAI's GPT-4.5, Anthropic's Claude 4, and xAI's Grok 3 all pushing the boundaries of what large language models can accomplish.

This rapid advancement has heightened concerns among policymakers, researchers, and even some industry insiders. Yoshua Bengio, a Turing Award–winning AI researcher, has repeatedly warned that the pace of development is outstripping society's ability to evaluate and mitigate risks.

Meanwhile, the private sector has built its own safety infrastructure. Anthropic has developed its Responsible Scaling Policy, which ties model deployment decisions to demonstrated safety levels. OpenAI maintains a safety advisory board and publishes system cards for major releases. Google DeepMind operates an internal safety team that has published influential research on AI alignment.

The government's role, then, is not to replace these internal efforts but to provide an independent layer of verification. Think of it as an external audit function — the companies conduct their own safety testing, and AISI provides an additional, neutral assessment.

What This Means for Developers, Businesses, and Users

For enterprise customers evaluating AI vendors, government safety testing adds a valuable signal. Organizations in regulated industries like healthcare, finance, and defense increasingly require evidence that AI systems have undergone rigorous third-party evaluation. A Commerce Department stamp of approval — even an informal one — could influence procurement decisions worth billions of dollars.

AI developers building on top of these foundation models should pay close attention to the findings AISI publishes. Safety evaluations often reveal failure modes and vulnerabilities that can inform better application design and risk mitigation strategies.

For everyday users, the practical impact may be less immediately visible but no less important. Government testing creates accountability pressure that incentivizes companies to invest in safety before releasing products. Even without legal mandates, the reputational cost of a negative government evaluation can be substantial.

The voluntary nature of the agreements does create limitations, however. Companies can theoretically withdraw from the pacts or selectively share information. Without congressional action to codify these arrangements into law, their durability depends entirely on the willingness of both the government and the companies to participate in good faith.

Looking Ahead: Will Voluntary Commitments Be Enough?

The central question hanging over these agreements is whether voluntary safety testing can keep pace with the exponential growth in AI capabilities. Several factors will determine the answer in the coming months and years.

Congressional action remains the most likely path to making these arrangements permanent. Multiple bipartisan AI safety bills are currently moving through committee, though none have gained sufficient momentum for a floor vote. The political dynamics are complex — lawmakers want to appear responsible on AI safety without alienating the tech industry or stifling American competitiveness against China.

The international dimension also looms large. The UK's AI Safety Institute, established after the November 2023 AI Safety Summit at Bletchley Park, has conducted its own evaluations of frontier models. Coordination between U.S. and international safety bodies could create a more comprehensive global testing regime, but geopolitical tensions — particularly with China — complicate these efforts.

Expect to see additional companies join these agreements in the coming quarters. Meta, which has taken an open-source approach with its Llama models, and Amazon, which has invested heavily in Anthropic, are likely candidates. The more companies that participate, the stronger the norm becomes — and the harder it gets for holdouts to justify staying on the sidelines.

For now, the Commerce Department's latest agreements represent a pragmatic middle ground in America's approach to AI governance: not as prescriptive as Europe's regulatory framework, but more structured than a pure laissez-faire approach. Whether that middle ground proves sufficient as AI systems grow ever more powerful remains one of the defining questions of the decade.