SK Hynix Unveils HBM4 Memory for AI Training

SK Hynix, the world's second-largest memory chipmaker, has officially announced the development of its next-generation HBM4 (High Bandwidth Memory 4) chips, purpose-built to meet the escalating demands of AI training workloads. The new memory standard promises a massive leap in bandwidth, capacity, and energy efficiency compared to the current HBM3E generation — positioning the South Korean semiconductor giant to dominate the rapidly expanding AI infrastructure market.

The announcement comes as demand for high-bandwidth memory has surged to unprecedented levels, driven primarily by NVIDIA, AMD, and other accelerator manufacturers racing to build more powerful AI training chips. SK Hynix currently supplies HBM3E chips for NVIDIA's H200 and upcoming B200 GPUs, and HBM4 is expected to power the next wave of AI accelerators arriving in 2025 and 2026.

Key Facts at a Glance

• Bandwidth boost: HBM4 is expected to deliver over 2 TB/s of memory bandwidth per stack, roughly double the throughput of HBM3E
• Capacity increase: Each HBM4 stack could reach 48 GB or higher, up from 24 GB in current HBM3E configurations
• Energy efficiency: SK Hynix targets a 30-40% improvement in performance per watt over HBM3E
• Manufacturing timeline: Mass production is projected to begin in the second half of 2025, with volume shipments ramping through 2026
• Primary customers: NVIDIA, AMD, and potentially Intel and custom ASIC designers like Google and Amazon
• Architecture: HBM4 adopts the JEDEC-standardized interface with a wider I/O bus of 2,048 bits per stack

HBM4 Doubles Down on Bandwidth and Capacity

The core innovation behind HBM4 lies in its dramatically expanded memory bus. While HBM3E operates with a 1,024-bit wide interface per stack, HBM4 doubles this to 2,048 bits. This architectural change is the primary driver behind the expected bandwidth leap to over 2 TB/s — a figure that would have seemed extraordinary just 2 years ago.

SK Hynix plans to achieve higher capacities by stacking 16 DRAM layers using advanced through-silicon via (TSV) technology. Current HBM3E products typically stack 8 or 12 layers. The move to 16 layers requires significant advances in die thinning, thermal management, and bonding precision — areas where SK Hynix has invested billions of dollars in R&D.

The company has also indicated that HBM4 will incorporate logic-in-memory capabilities, embedding basic computational functions directly within the memory stack. This approach reduces the amount of data that needs to travel between the memory and the GPU, cutting latency and power consumption simultaneously.

Why AI Training Demands Keep Outpacing Memory Technology

Modern large language models (LLMs) like OpenAI's GPT-4, Google's Gemini, and Meta's Llama 3 require enormous amounts of memory bandwidth during training. A single training run for a frontier model can involve tens of thousands of GPUs, each needing to shuttle massive datasets between compute cores and memory at blistering speeds.

The bottleneck is no longer just compute power — it is increasingly memory bandwidth. NVIDIA's current H100 GPU, equipped with 80 GB of HBM3 memory delivering 3.35 TB/s aggregate bandwidth, still leaves researchers waiting for data transfers during training. The upcoming B200 GPU with HBM3E improves this, but industry analysts expect even that to become a constraint as models scale toward 10 trillion parameters and beyond.

HBM4 addresses this gap directly. By offering roughly 2x the bandwidth of HBM3E at the per-stack level, it enables GPU designers to build accelerators that can feed data to their compute engines without starving them. For AI companies spending $100 million or more on a single training run, even marginal improvements in memory throughput translate to millions of dollars in saved compute time.

SK Hynix Strengthens Its Grip on the AI Memory Market

SK Hynix currently commands an estimated 50% share of the global HBM market, with Samsung and Micron Technology splitting most of the remainder. The company's early and aggressive bet on HBM technology — dating back to its pioneering work on HBM1 in 2015 — has given it a substantial manufacturing and yield advantage.

The financial results reflect this dominance. SK Hynix reported record quarterly revenue in Q1 2024, with HBM sales accounting for a rapidly growing share of total DRAM revenue. Analysts at Morgan Stanley estimate that HBM could represent over 25% of SK Hynix's total DRAM revenue by the end of 2025, up from roughly 10% in 2023.

Key competitive advantages for SK Hynix include:

• Yield leadership: Higher manufacturing yields on HBM3E compared to Samsung, translating to lower costs per chip
• Customer lock-in: Deep engineering partnerships with NVIDIA, including co-design of memory interfaces
• Packaging expertise: Advanced CoWoS-like packaging capabilities through its partnership with TSMC
• Capacity investment: Over $10 billion earmarked for HBM and advanced DRAM expansion through 2028

Samsung has been working aggressively to close the gap, recently announcing its own HBM4 roadmap and investing heavily in yield improvements. Micron, while a smaller player in HBM, has also signaled plans for next-generation products. The race is intensifying, but SK Hynix's head start remains significant.

How HBM4 Compares to Current Memory Standards

Understanding HBM4's significance requires context on how rapidly this technology has evolved. The progression from HBM2E to HBM4 represents a roughly 4x improvement in bandwidth and a 3x improvement in capacity in just 4 years.

Compared to traditional DDR5 memory used in standard servers, HBM4 offers roughly 10x the bandwidth in a fraction of the physical footprint. This density advantage is critical for AI accelerators, where board space is at a premium and every millimeter counts.

What This Means for the AI Industry

For AI companies and cloud providers, HBM4 represents a meaningful step toward training larger, more capable models without proportionally increasing infrastructure costs. Higher memory bandwidth means fewer GPUs needed for the same training throughput, which directly impacts the economics of AI development.

For GPU and accelerator designers, HBM4 opens new architectural possibilities. NVIDIA's next-generation 'Rubin' platform, expected in 2026, is widely anticipated to adopt HBM4. AMD's Instinct MI-series accelerators and custom AI chips from Google (TPUs), Amazon (Trainium), and Microsoft (Maia) could also benefit from the new standard.

For investors and the broader semiconductor industry, HBM4 reinforces a critical trend: memory is becoming as strategically important as logic chips in the AI era. The days when DRAM was viewed as a commoditized, cyclical business are fading fast. Memory companies with HBM expertise now command premium valuations and enjoy pricing power that was unimaginable a decade ago.

Looking Ahead: The Road to HBM4 and Beyond

SK Hynix has indicated that engineering samples of HBM4 will be available to key customers by late 2025, with mass production scaling through 2026. The company is already exploring HBM4E, an enhanced variant that could push bandwidth beyond 3 TB/s per stack — though details remain scarce.

Several challenges remain on the path to HBM4 adoption. Thermal management becomes increasingly difficult as more DRAM layers are stacked, and the wider I/O bus requires tighter integration with GPU packaging technologies. SK Hynix and its packaging partners, including TSMC, will need to solve these engineering problems at scale.

The broader trajectory is clear, however. As AI models continue to grow in size and complexity, the appetite for memory bandwidth shows no signs of slowing. SK Hynix's early move on HBM4 positions it to capture a disproportionate share of what analysts at Bank of America project will be a $25+ billion HBM market by 2027, up from roughly $4 billion in 2023.

In the AI chip arms race, the memory makers are no longer supporting actors — they are co-stars. And SK Hynix, with HBM4, is writing itself into the lead role.