EU AI Act: The Case for Foundation Model Impact Assessments

The EU AI Act is now the world's most comprehensive AI regulation, but a critical gap remains: it does not explicitly mandate full algorithmic impact assessments (AIAs) for foundation models. As companies like OpenAI, Google, Meta, and Anthropic deploy increasingly powerful systems across Europe, policymakers, researchers, and civil society groups are pressing for mandatory pre-deployment evaluations that go far beyond current requirements.

This debate is not academic. It will shape how $200+ billion in global AI investment flows over the next decade — and whether Europe leads in responsible AI governance or creates friction that pushes innovation elsewhere.

Key Takeaways

• The EU AI Act classifies foundation models under general-purpose AI (GPAI) rules but stops short of requiring full algorithmic impact assessments
• Unlike Canada's proposed Artificial Intelligence and Data Act (AIDA), the EU framework lacks a standardized AIA template for foundation models
• OpenAI, Google DeepMind, and Anthropic currently publish voluntary 'model cards' and 'system cards,' but critics say these lack rigor and accountability
• An estimated 85% of AI-related harms stem from downstream deployment contexts that foundation model developers may not anticipate
• The EU AI Office, operational since early 2024, is drafting codes of practice that could effectively introduce AIA-like obligations
• Compliance costs for mandatory AIAs could range from $500,000 to $5 million per model, according to industry estimates

What the EU AI Act Currently Requires

The EU AI Act, which entered into force in August 2024 with phased enforcement through 2027, creates a tiered system. High-risk AI systems — those used in hiring, credit scoring, law enforcement, and healthcare — face the strictest obligations, including conformity assessments, human oversight requirements, and detailed documentation.

Foundation models fall under the GPAI provisions in Articles 51–56. Providers of GPAI models must maintain technical documentation, comply with EU copyright law, and publish training data summaries. Models posing 'systemic risk' — currently defined as those trained with more than 10^25 FLOPs of compute — face additional obligations including adversarial testing and incident reporting.

However, none of these requirements constitute a full algorithmic impact assessment. An AIA, as understood in policy literature, involves a structured evaluation of a system's potential societal impacts — including effects on civil rights, discrimination, privacy, democratic processes, and environmental sustainability — conducted before deployment and updated regularly.

Why Foundation Models Present Unique Challenges

Traditional algorithmic impact assessments were designed for narrow AI systems with well-defined use cases. A facial recognition system deployed by police has a clear context, affected population, and risk profile. Foundation models break this paradigm entirely.

GPT-4, Claude 3.5, Gemini 1.5, and Llama 3 are general-purpose systems that can be adapted for thousands of downstream applications. When OpenAI releases a new model, it cannot predict every way developers and enterprises will use it. This creates what researchers call the 'assessment gap' — the distance between what a model provider can reasonably evaluate and the full spectrum of real-world impacts.

Key challenges include:

• Unpredictable downstream uses: A language model built for customer service might be fine-tuned for medical diagnosis, legal advice, or political content generation
• Emergent capabilities: Models display behaviors at scale that were not present or tested during development
• Cascading effects: Foundation models are embedded in complex software stacks where their outputs interact with other systems
• Cross-border deployment: A model trained in the US and hosted on global cloud infrastructure serves users across dozens of legal jurisdictions simultaneously
• Rapid iteration cycles: Major labs release updated models every 3–6 months, potentially outpacing any assessment framework

These challenges do not argue against impact assessments — they argue for a fundamentally redesigned approach.

The Case for Mandatory Assessments

Proponents of mandatory AIAs for foundation models point to several compelling arguments. The precautionary principle, deeply embedded in European regulatory philosophy, suggests that systems with potentially massive societal impacts should be evaluated before widespread deployment, not after harm occurs.

The AI Now Institute at New York University has long advocated for mandatory impact assessments, arguing that voluntary self-reporting creates an inherent conflict of interest. When OpenAI publishes a system card for GPT-4, it simultaneously serves as the model's developer, evaluator, and marketer. Independent, structured assessments would separate these roles.

Research from the Alan Turing Institute in the UK suggests that structured impact assessments can identify 60–70% of foreseeable harms before deployment. While not perfect, this represents a significant improvement over current practices, where many risks are discovered only after millions of users interact with a system.

Canada's proposed AIDA framework offers a potential template. It would require 'high-impact' AI systems to undergo assessments that evaluate bias, privacy implications, and effects on vulnerable populations. Adapting this approach for foundation models would require modifications, but the underlying principle — structured, mandatory evaluation with public accountability — translates directly.

Furthermore, mandatory AIAs could level the competitive playing field. Currently, companies like Anthropic invest heavily in safety evaluations, while smaller competitors may cut corners. A standardized requirement ensures all market participants meet baseline safety standards, rewarding rather than penalizing responsible development.

The Case Against — and Industry Pushback

Critics of mandatory AIAs for foundation models raise legitimate concerns. The most fundamental objection is practical: how do you assess the impact of a system whose applications are, by definition, open-ended?

Industry groups like DigitalEurope and the Computer & Communications Industry Association (CCIA) have argued that overly prescriptive assessment requirements could stifle innovation, particularly for European AI startups like France's Mistral AI and Germany's Aleph Alpha. These companies, already competing against better-funded American rivals, warn that compliance costs of $500,000 to $5 million per model could be prohibitive.

There are also concerns about assessment quality and 'checkbox compliance.' If AIAs become formulaic, companies might treat them as bureaucratic exercises rather than genuine safety evaluations. The history of environmental impact assessments — which are sometimes criticized as rubber-stamp processes — offers a cautionary tale.

Additional counterarguments include:

• Speed of development: Quarterly model releases may outpace any meaningful assessment timeline
• Trade secrets: Detailed impact assessments could require disclosing proprietary training methods and data
• Regulatory fragmentation: Different AIA standards across jurisdictions could create conflicting obligations
• False sense of security: A passed assessment might create unwarranted confidence in a model's safety
• Resource diversion: Funds spent on compliance might be better directed toward actual safety research

Some industry leaders have proposed a middle ground: mandatory assessments for models above certain capability thresholds, with lighter requirements for smaller or more specialized systems.

How Other Jurisdictions Are Approaching This

The United States has taken a largely voluntary approach. President Biden's October 2023 Executive Order on AI encouraged but did not mandate impact assessments for foundation models. The National Institute of Standards and Technology (NIST) published its AI Risk Management Framework, which provides guidelines but carries no legal force.

Compared to the EU's regulatory approach, the US model relies on industry self-governance and market incentives. This creates an interesting natural experiment: will Europe's stricter approach produce safer AI systems, or will it simply push development to less regulated markets?

China's approach is perhaps the most prescriptive. The Cyberspace Administration of China requires generative AI services to undergo security assessments before public launch, including evaluations of training data quality and content safety. However, these assessments focus primarily on political content control rather than broader societal impact.

The UK, post-Brexit, has positioned itself as a 'pro-innovation' alternative to the EU, with its AI Safety Institute conducting evaluations of frontier models on a voluntary basis. The institute has evaluated models from OpenAI, Anthropic, Google DeepMind, and Meta, but participation remains optional.

What a Practical AIA Framework Could Look Like

Designing an effective AIA framework for foundation models requires rethinking traditional approaches. Several research groups have proposed tiered assessment models that balance thoroughness with practicality.

A workable framework might include these elements:

• Pre-deployment technical evaluation: Red-teaming, bias auditing, and capability benchmarking conducted by both the developer and independent third parties
• Societal impact mapping: Structured analysis of potential effects on civil rights, labor markets, information ecosystems, and vulnerable populations
• Ongoing monitoring obligations: Continuous post-deployment assessment with mandatory incident reporting and periodic reassessment
• Public transparency reports: Standardized, publicly accessible summaries of assessment findings — not just developer-authored model cards
• Downstream responsibility sharing: Clear frameworks for allocating assessment responsibilities between foundation model providers and downstream deployers

The EU AI Office is currently developing codes of practice for GPAI providers, expected to be finalized by mid-2025. These codes represent the most immediate opportunity to introduce AIA-like requirements without reopening the legislative text of the AI Act itself.

What This Means for Developers and Businesses

For AI developers operating in or serving the European market, the trajectory is clear: more structured impact evaluation is coming, whether through formal legislation, codes of practice, or market expectations.

Practical steps companies should consider now include investing in internal assessment capabilities, engaging with the EU AI Office's code of practice consultations, and building documentation practices that could satisfy future AIA requirements. Companies that treat impact assessment as a core competency — rather than a compliance burden — will be better positioned regardless of the regulatory outcome.

For businesses deploying foundation models in their products, the key question is liability allocation. If mandatory AIAs are introduced, will downstream deployers be responsible for assessing their specific use cases, or will foundation model providers bear the primary obligation? The answer will significantly affect build-versus-buy decisions across the enterprise AI market.

Looking Ahead: The Next 18 Months Are Critical

The window for shaping foundation model governance in Europe is narrowing. The EU AI Office's codes of practice, expected by mid-2025, will set the practical standard for GPAI compliance. The full enforcement of GPAI provisions begins in August 2025, creating an immediate deadline for providers.

Meanwhile, the political landscape is shifting. European Parliament members from multiple parties have called for stronger foundation model oversight, and civil society organizations are preparing legal challenges that could test the adequacy of current provisions.

The most likely outcome is a pragmatic compromise: structured assessment requirements that are meaningful but scaled to model capability, with heavier obligations for frontier systems and lighter requirements for smaller models. This approach mirrors the EU AI Act's existing risk-based philosophy and could serve as a global template.

What remains uncertain is whether such a framework can keep pace with AI development. If major labs continue releasing substantially more capable models every 6–12 months, any assessment regime risks becoming obsolete before it is fully implemented. The ultimate success of algorithmic impact assessments for foundation models will depend not just on regulatory design, but on building institutional capacity to evaluate systems that grow more complex with each generation.