Gemini Ultra 2 Matches Humans on Grad Math Exams

Google DeepMind has unveiled Gemini Ultra 2, its most advanced large language model to date, which achieves human-level performance on graduate-level mathematics examinations for the first time. The model scored an average of 89.7% across a suite of PhD-qualifying math exams, matching or exceeding the median scores of human graduate students in fields ranging from abstract algebra to differential topology.

This breakthrough represents a seismic shift in AI reasoning capabilities and puts Google in a commanding position in the intensifying race against OpenAI, Anthropic, and Meta to build models that can genuinely think through complex problems rather than simply pattern-match from training data.

Key Takeaways at a Glance

• Gemini Ultra 2 scored 89.7% on graduate math exams, up from 68.4% achieved by the original Gemini Ultra
• The model solved problems in real analysis, abstract algebra, topology, and combinatorics at human-expert levels
• DeepMind credits a new chain-of-verification reasoning architecture for the leap in mathematical performance
• Performance on the MATH benchmark reached 96.4%, surpassing OpenAI's o3 model score of 93.2%
• The model will be available through Google Cloud's Vertex AI platform at $0.015 per 1,000 input tokens
• A research paper detailing the technical approach has been submitted to NeurIPS 2025

How Gemini Ultra 2 Cracks Graduate-Level Math

The secret behind Gemini Ultra 2's mathematical prowess lies in what DeepMind calls chain-of-verification (CoV), a novel reasoning framework that extends the chain-of-thought paradigm popularized by OpenAI's o1 and o3 models. Unlike traditional chain-of-thought, which generates a linear sequence of reasoning steps, CoV introduces a self-auditing loop where the model independently verifies each intermediate step before proceeding.

In practice, this means Gemini Ultra 2 doesn't just solve a differential equation — it checks its own work at every stage. DeepMind's researchers found that this approach reduced logical errors by 67% compared to the original Gemini Ultra and by 43% compared to standard chain-of-thought methods.

The model was evaluated on a custom benchmark called GradMath, which comprises 1,247 problems drawn from actual qualifying exams at 15 top-tier mathematics departments, including MIT, Princeton, and Cambridge. Problems span 8 subfields of pure and applied mathematics, each requiring multi-step formal reasoning that has historically stumped AI systems.

Benchmark Results Reveal Dominant Performance

The numbers tell a compelling story. Gemini Ultra 2 doesn't just inch past its competitors — it establishes clear leads across virtually every major mathematical reasoning benchmark.

• MATH benchmark: 96.4% (vs. OpenAI o3 at 93.2%, Claude 3.5 Opus at 88.1%)
• GSM8K: 99.1% (effectively solved, matching several frontier models)
• GradMath (new): 89.7% (median human graduate student score: 87.3%)
• AIME 2024 problems: Solved 14 out of 15 correctly (OpenAI o3 solved 12)
• Putnam Competition problems: Solved 4 out of 6 selected problems from the 2023 exam
• Formal proof generation: Successfully generated Lean 4 proofs for 34% of tested theorems, up from 11% with Gemini Ultra

What makes these results particularly striking is the performance on the Putnam Competition, widely considered one of the hardest undergraduate math competitions in North America. Solving 4 out of 6 Putnam problems places Gemini Ultra 2 in roughly the top 5% of human participants — a feat that would have seemed impossible just 2 years ago.

The Technical Architecture Behind the Leap

DeepMind has not disclosed the full parameter count of Gemini Ultra 2, but industry analysts estimate it falls in the range of 1.5 to 2 trillion parameters, making it one of the largest dense models ever deployed. The model builds on Google's Mixture of Experts (MoE) architecture but introduces several key innovations.

First, the training pipeline incorporated what DeepMind describes as synthetic theorem proving, where the model was trained on millions of machine-generated mathematical proofs verified by formal proof assistants like Lean 4 and Isabelle. This approach ensures that the training data contains only logically valid reasoning chains, unlike web-scraped math content which often contains errors.

Second, DeepMind implemented a reward model specifically tuned for mathematical correctness. Rather than relying solely on human preference data — the standard approach in RLHF — this reward model was trained to evaluate the logical validity of each reasoning step. The result is a system that optimizes for truth rather than plausibility.

Third, the model features an expanded context window of 2 million tokens, allowing it to process entire textbooks or lengthy proofs without losing track of earlier definitions and lemmas. This represents a 2x increase over the original Gemini Ultra's 1 million token context window.

Industry Context: The Math Reasoning Arms Race Heats Up

Gemini Ultra 2's achievement arrives at a pivotal moment in the AI industry. Mathematical reasoning has become the key battleground for frontier AI labs, as it serves as a proxy for general logical thinking and problem-solving ability.

OpenAI set the pace in late 2024 with its o1 and o3 reasoning models, which demonstrated that extended 'thinking time' could dramatically improve performance on math and coding tasks. Anthropic followed with Claude 3.5 Opus, which introduced its own extended thinking capabilities. Meta's Llama 4 Behemoth has also shown strong mathematical reasoning in early benchmarks.

But DeepMind's results suggest that Google has leapfrogged the competition, at least on mathematical tasks. The 3.2 percentage point lead over OpenAI's o3 on the MATH benchmark is significant — in a domain where top models are separated by fractions of a percent, this gap represents a meaningful technical advantage.

Industry observers note that mathematical reasoning capability is increasingly viewed as a leading indicator of broader AI competence. Models that can handle rigorous mathematical proofs tend to perform better at scientific research, financial modeling, and engineering design — all high-value commercial applications.

What This Means for Developers and Businesses

For the developer community and enterprise customers, Gemini Ultra 2's capabilities open up several practical possibilities that were previously out of reach.

Scientific research teams can now use AI to verify complex mathematical proofs, potentially accelerating peer review and reducing errors in published papers. Several mathematics departments have already expressed interest in using the model as a 'proof assistant' for graduate students.

Financial institutions stand to benefit from the model's improved ability to handle quantitative modeling. Goldman Sachs and JPMorgan Chase have both reportedly begun testing Gemini Ultra 2 for derivatives pricing and risk analysis applications.

Engineering firms can leverage the model for complex simulation and optimization problems that require multi-step mathematical reasoning. Early testers report that the model can set up and solve partial differential equations relevant to structural engineering and fluid dynamics with minimal human guidance.

Pricing is set at $0.015 per 1,000 input tokens and $0.06 per 1,000 output tokens through Vertex AI, positioning it competitively against OpenAI's o3 pricing of $0.02 per 1,000 input tokens. Google is also offering a 30-day free trial with $500 in credits for enterprise customers.

Limitations and Caveats Worth Noting

Despite the impressive results, researchers urge caution in interpreting the findings. Several important limitations remain.

Gemini Ultra 2 still struggles with novel conjectures — problems that require genuine mathematical creativity rather than applying known techniques. On a set of 50 open research problems curated by Fields Medal winners, the model failed to make meaningful progress on any of them.

The model also shows inconsistency on certain problem types. When tested on the same problem rephrased in different ways, it occasionally produced contradictory answers, suggesting that its reasoning is not always as robust as the headline numbers imply.

Additionally, the GradMath benchmark has not yet been independently verified by third parties. Some researchers on social media have raised concerns about potential data contamination — the possibility that some exam problems may have appeared in the model's training data. DeepMind says it took extensive precautions to prevent this, including using only exams administered after the model's training data cutoff.

Looking Ahead: The Road to Mathematical Discovery

DeepMind CEO Demis Hassabis has framed Gemini Ultra 2 as a stepping stone toward AI systems capable of genuine mathematical discovery — not just solving known problems, but formulating and proving new theorems. The company's AlphaProof project, which combines Gemini with formal verification systems, is reportedly making progress toward this goal.

The timeline for broader availability is aggressive. Google plans to integrate Gemini Ultra 2's reasoning capabilities into Bard Advanced by Q3 2025, making graduate-level math assistance available to consumers for a $29.99 monthly subscription. A distilled version optimized for mobile devices is expected by early 2026.

For the AI industry at large, this milestone raises profound questions. If AI systems can match human experts on graduate-level mathematics today, what does the next 2 to 3 years hold? The consensus among leading researchers is that mathematical discovery — not just problem-solving — could be within reach by 2027.

The race to build AI that can truly reason is no longer theoretical. With Gemini Ultra 2, Google DeepMind has fired a shot that will reverberate across the entire industry — and across every field that depends on mathematical thinking.