SK hynix announced HBM3’s development in October 2021, with the company beginning to mass produce the product in June 2022. SK hynix will also provide HBM3 for NVIDIA systems that are expected to begin shipping in the third quarter of 2022.

How did the company maintain its leadership position in this market, and what lessons did it implement from developing the previous generations of HBM products?

HBM1 – First Mover from the Get-go

When HBM was developed as a memory solution optimized for high-level computing performance (HCP), it offered a new paradigm on solving the memory bottleneck as it aimed to increase capacity and bandwidth simultaneously. SK hynix jointly developed the world’s first TSV (Through Silicon Via) HBM product with AMD in 2014. The two companies also teamed up to develop high-bandwidth 3-D stacked memory technologies and related products.

HBM1’s operating frequency is around 1,600 Mbps, the VDD (drain power voltage) is 1.2V, and the die density is 2Gb (4-hi). The product had a higher bandwidth than the DDR4 and GDDR5 products, while using less power in a substantially smaller form factor, benefitting bandwidth-hungry processors such as GPUs (graphics processing units).

HBM2 – Second Generation Home Improvements

In the second generation HBM2, released in 2018, a key improvement was the Pseudo Channel mode.  This mode divides a channel into two separate 64-bit I/O sub-channels, as well as providing 128-bit prefetch per memory read and write access for each sub-channel. The mode optimizes memory accesses and lowers latency, resulting in higher effective bandwidth.

Other improvements included lane remapping modes for both hard and soft repairs of lanes, as well as anti-overheating protection. The newer technologies, alongside HBM2’s higher effective bandwidth, give it a higher energy efficiency than HBM1 at its data-rate.

HBM2E – Round Three Game Changer

SK hynix was also the first memory vendor to begin mass producing HBM2E, an extended version of HBM2. The HBM2E development team’s determination to raise the product’s specifications at the planning stage played a critical role in helping SK hynix to maintain its leadership position. The product was released two years after HBM2 in 2020, with technological updates and more applications, as well as a faster speed and higher capacity than HBM2.

The product’s 16Gb die density was double that of HBM2, achieved by vertically stacking eight 16Gb chips via TSV technology. At the time of its release, HBM2E had the industry’s fastest memory at 3.6Gbps in I/O speed and processing 460GB of data per second using 1,024 I/Os. HBM2E also has 36% better heat dissipation than HBM2.

HBM3 – Maintaining Leadership into the Fourth Generation

SK hynix continued maintaining its leadership status with HBM3, which was the world’s first of its kind when the company developed it in 2021. HBM3 has 1.5 times HBM2E’s capacity from 12 DRAM die stacked to the same total package height, enabling it to power capacity-intensive applications such as AI and HPC.

A significant addition in HBM3 compared to previous generations is a custom-designed on-die ECC (Error Correcting Code), which uses pre-allocated parity bits to check and correct errors in the data received. The code also allows DRAM to self-correct errors within cells, enhancing device reliability.

HBM3’s 16-channel architecture runs at 6.4Gbps, double that of HBM2E and currently the fastest in the world. This makes HBM3 and other HBM products an indispensable component of digital life, such as HBM products becoming a prerequisite for the Levels 4 and 5 of driving automation in autonomous vehicles.

SK hynix’s HBM development is also an important component of the company’s ESG efforts, with each generation of the product designed to consume less power than the previous one. For example, HBM3 runs at lower temperatures than HBM2E at the same level of operating voltage, enhancing the stability of the server system environment, and increasing cooling capacity.

After celebrating its HBM3 achievements, the development team has already moved onto the next step, cooperating with clients, and receiving feedback on the product. Predictions are also already out for HBM4, which could be more widely used in areas such as high-performance data centers, super computers, and artificial intelligence.

The HBM market also continues its steady growth, with the volume of data transmission increasing rapidly in the 5G era and a 2021 report by Omdia predicting that the market will generate $2.5 billion in revenue by 2025. SK hynix is looking to secure its leadership in the market by continuing to take its HBM products to the next level and maintaining its position as not only a “first mover”, but also a “solution provider”.

The post Continuing to Make HBM History: The Story of SK hynix’s HBM Development first appeared on SK hynix Newsroom.

Through-Silicon-Via (TSV) in memories has emerged as an efficient foundational technology for capacity expansion and bandwidth extension. It is a technology where vias are perforated through the entire silicon wafer thickness, in order to form thousands of vertical interconnections from the front to the back-side of the die and vice versa. In the earlier days, TSV was regarded merely as a packaging technology, simply replacing wire-bondings. But over the years, it has become an essential tool to scale performance and density of DRAMs. Today, there are two main use cases in the DRAM industry, where TSVs have been successfully productized to overcome capacity and bandwidth scaling limitations. These are 3D-TSV DRAM and High-Bandwidth-Memory (HBM).

Image Download

High density memories such as 128 and 256GB DIMMs (16Gb based 2rank DIMMs with 2High and 4High X4 DRAMs) are also adopting 3D-TSV DRAMs in addition to traditional Dual-Die-Packages (DDP) having wire-bonded die stacks. In 3D-TSV DRAMs, 2 or 4 DRAM dies are stacked on top of each other, where only the bottommost die is connected externally to the memory controller. The remaining dies are interconnected through many TSVs internally providing Input/Output (I/O) load isolations. Compared to DDP structures, such architecture allows higher pin speeds through decoupling of I/O loadings, and lower power consumption by eliminating unnecessary duplication of circuit components across the stacked dies.

Image Download

On the other hand, HBM has been primarily created to bridge the bandwidth gap between the system- on-a-chip (SoC) high bandwidth demands and maximum bandwidth supply capabilities of main memories. For example, in AI applications, bandwidth demands per SoC, particularly in training applications, can exceed a few TB/s which cannot be satisfied with conventional main memories. A single main memory channel with 3200Mbps DDR4 DIMMs can provide only 25.6GB/s bandwidth. Even the most advanced CPU platforms with 8 memory channels can provide only 204.8GB/s. On the other hand, 4 HBM2 stacks around a single SoC can provide >1TB/s bandwidth, capable of bridging their bandwidth gaps. Depending on applications, HBMs can be used either stand-alone, as a cache or as the first tier in a two-tier memory.

HBM is an in-package memory where it is integrated with a SoC through a silicon interposer inside the same package. This allows it to overcome the maximum number of data I/O package pin limitations, which would otherwise exist in conventional off-chip packages. HBM2, which has already been deployed in actual products, consists of 4 or 8-high stacks of 8Gb dies and 1024 data pins running at 1.6~2.4Gbps each. This results in 4 or 8GB density and 204~307GB/s bandwidth per HBM stack.

Image Download

SK hynix has always been committed to maintaining industry leadership in both HBM and high-density 3D-TSV DRAM products. Recently, SK hynix announced the successful development of the HBM2E device, an extended version of HBM2, which reaches higher density of up to 16GB and bandwidth of 460GB/s per stack. This was made possible through the increase of DRAM die density to 16Gb and achieving 3.6Gbps per pin speed over 1024 data IOs at 1.2V supply voltage. SK hynix is also in the process of expanding line-ups of 128~256GB 3D-TSV DIMMs to satisfy the needs of its customers for higher density DIMMs.

TSV technology has now reached its maturity to some extent, being able to build state-of-the-art products such as HBM2Es with thousands of TSVs. In the future, however, decreasing the TSV pitch¹/diameter/aspect-ratio² and the die thickness, while still maintaining high assembly yields, will become more challenging and essential for continued future device performance and capacity scaling. Such improvements will allow decreased TSV loadings, reduced TSV relative die size portions, and the extended number of stacks beyond 12Highs while still maintaining the same total physical stack height. SK hynix will continue to be dedicated to positioning itself at the forefront of memory technology leadership through endless innovations in TSV products and technology.

¹The distance between two vias
²The ratio of the height to the diameter of the TSV

The post Creating New Values in DRAM Using Through-Silicon-Via Technology for Continued Scaling in Memory System Performance and Capacity first appeared on SK hynix Newsroom.