The world runs on data—a ceaseless tide of ones and zeros surging through networks, powering everything from streaming services to AI. To handle this data deluge, data centers must employ more advanced memory solutions that meet ever-growing performance demands.

However, traditional methods of scaling memory are facing limitations. The constraints of processors and memory technologies, coupled with escalating costs and power consumption for data centers, have highlighted the need for a revolutionary approach. Enter Compute Express Link^® (CXL^®), a transformative memory interconnect technology designed to tackle the challenges of the AI era.

This Rulebreakers’ Revolutions  episode will cover SK hynix’s development of CXL solutions, detailing how the company overcame obstacles, such as a lack of industry specifications, to become a leader in the CXL field and a key contributor to the CXL ecosystem.

Mission: Harnessing New Interconnect Tech for Memory Scaling

In the AI era, data centers need to continuously expand their memory capacity to handle ever-growing volumes of data. However, scaling memory capacity through traditional methods is becoming prohibitively expensive and inefficient. For example, adding terabyte (TB)-scale memory to a single CPU system can significantly increase the total cost of ownership¹ (TCO) and power consumption. Attempting to address this by increasing memory channels or integrating higher capacity memory often results in even greater power usage and heat generation, further inflating cooling system and management costs. This has underlined the need for innovative memory system designs which can process data faster, more efficiently, and cost effectively.

¹Total cost of ownership: The complete cost of acquiring, operating, and maintaining an asset, including purchase, energy, and maintenance expenses.

Data center memory capacity needs to increase to handle the growing demands of the AI era

Over the past decade, the industry has been developing new memory interconnect technologies to meet this market demand. Memory interconnect technology refers to the method by which processors exchange data with memory, playing a critical role in determining the speed and efficiency of data processing. In traditional memory architectures, memory is physically connected to a nearby single processor, which can lead to an over-provision of memory resources when applications are not using the memory. New memory interconnect technologies such as CXL can overcome this issue by allowing multiple processors to share memory for improved efficiency.

This has led to great interest in CXL, however developing the technology would prove challenging as there was no precedent for the process and initially no industry-established specifications. Without the JEDEC² specifications which are generally provided for DRAM products, the development process for CXL was fundamentally more complex than usual.

²JEDEC Solid State Technology Association: With over 350 member companies, JEDEC is the global leader in developing open standards for the microelectronics industry.

Faced with having to develop new CXL products without industry specifications to break the barriers of memory scaling, SK hynix tapped into its internal expertise and collaborated with industry partners.

Into the Unknown: Developing Pioneering CXL Tech From Scratch

Following the introduction of CXL in 2019, SK hynix soon recognized the technology’s capability to meet ever-growing memory scaling needs. As an open industry-standard interconnect, CXL unifies the interfaces of different system devices such as memory, storage, and processors. It supports features such as memory sharing, allowing multiple processors to access the same memory for improved data sharing, and memory pooling, in which memory from a common pool is assigned to processors for enhanced efficiency. Furthermore, CXL also enables memory switching, allowing hundreds of devices such as processors to share memory resources while independently processing data.

In addition to these innovative features, SK hynix became further convinced of CXL’s immense potential after observing increased market and customer commitment to the technology and identifying its promise in addressing technical and cost challenges. However, the company had to begin the project by overcoming a significant obstacle—a lack of industry specifications. SK hynix therefore soon set about developing its own basic requirement document after participating in CXL standardization activities and working with customers to define specifications. The company also collaborated with CXL controller companies to define controller requirements for the specifications document. Furthermore, the company has worked with JEDEC and the CXL Consortium³ to enhance DRAM-related specifications for industry CXL standards.

³CXL Consortium: An open industry standards group that develops technical specifications for CXL.

SK hynix’s CXL technology overcomes memory scaling challenges by expanding system capacity and bandwidth

Having helped set industry standards and develop relevant specifications, the company has accelerated its CXL development. For this process, SK hynix has identified key criteria to meet customer requirements—cost efficiency, high capacity, optimized bandwidth, and reliability.

First, cost efficiency is paramount in CXL development. To counteract the high cost of CXL controllers, it is crucial to minimize the costs of memory media such as modules. As high capacity is essential to facilitate large-scale data processing, the company determined CXL memory should offer storage two to four times larger than existing DDR products. Furthermore, bandwidth design must be optimized to utilize the full performance potential of CXL modules. Finally, reliability and data integrity must match the high standards of the host memory to earn customer trust.

To meet these criteria, multiple departments across SK hynix are working on making terabyte-scale memory more affordable and efficient. This includes pioneering memory pooling technologies that enable resource sharing among multiple devices and developing NMP⁴ technologies to handle data close to its source. These innovations are poised to deliver significant benefits to applications such as high-performance computing (HPC), in-memory databases, and AI.

⁴Near-memory processing (NMP): A technique that performs computations near data storage, reducing latency and boosting performance in high-bandwidth tasks like AI and HPC.

Through these efforts, SK hynix has been able to advance the development of groundbreaking CXL products which are set to revolutionize the memory field.

SK hynix’s Growing Product Lineup Driving the Future of CXL

Since developing its first DDR5-based CXL sample in 2022, SK hynix has been strengthening its CXL portfolio which includes the innovative CXL Memory Module-Double Data Rate 5 (CMM-DDR5). Leveraging high-speed PCIe Gen5 connections, CMM-DDR5 ensures smooth and rapid data processing. Available with up to 128 GB of storage, CMM-DDR5 also offers the high capacity required for the demands of today’s AI and HPC applications. In addition, the module boasts high levels of power efficiency and security.

Real-world performance tests highlight the transformative impact of CMM-DDR5. The product can expand system bandwidth by up to 82% and capacity by up to 100% compared to systems equipped with only DDR5 DRAM. Tests also showed how AI workloads experienced a 31% increase in token per second performance and that HPC enjoyed a 33% improvement in throughput efficiency. As well as providing outstanding performance, CMM-DDR5 has aligned with both JEDEC and CXL Consortium standards. Currently, the verification and certification of CMM-DDR5 is being carried out by customers as the product moves closer to mass production.

SK hynix’s CXL-based CMM-DDR5 enhances AI and HPC performance

SK hynix’s other CXL solutions include Niagara 2.0, an integrated hardware and software solution that allows multiple hosts to efficiently share large memory pools to minimize unused or underutilized memory. Furthermore, CXL Memory Module-Ax (CMM-Ax), a high-performance memory module optimized for computational workloads, is notable for improving AI and data center efficiency.

Beyond hardware advancements, SK hynix has developed the Heterogeneous Memory Software Development Kit (HSMDK) to maximize the potential of its CXL memory. This software toolkit has even been integrated into Linux’s operating system, further enhancing its accessibility and usability. The development of both hardware and software solutions as well as its standardization efforts highlights how SK hynix is committed to creating a thriving CXL ecosystem.

Rulebreaker Interview: “Thomas” Wonha Choi, Next-Gen Memory & Storage

In an interview with the SK hynix Newsroom, Distinguished Engineer⁵ (DE) “Thomas” Wonha Choi of Next-Gen Memory & Storage discussed the company’s rulebreaking mentality for developing CXL technology. Responsible for standardization efforts with JEDEC and the CXL Consortium and pathfinding next-generation memory such as CXL, Choi spoke about CXL’s development and its future impact.

⁵Distinguished Engineer: Senior SK hynix engineers who excel in their fields and are tasked with solving technical challenges and mentoring the next generation.

<Other articles from this series>

[Rulebreakers’ Revolutions] How MR-MUF’s Heat Control Breakthrough Elevated HBM to New Heights

[Rulebreakers’ Revolutions] How SK hynix Broke Barriers in Mobile DRAM Scaling With World-First HKMG Application

[Rulebreakers’ Revolutions] Innovative Design Scheme Helps HBM3E Reach New Heights

[Rulebreakers’ Revolutions] How SK hynix’s Server DRAM Validation Process Succeeds in a Diverse Server CPU Market

[Rulebreakers’ Revolutions] Flexible & Collaborative eSSD Virtualization Development for Today’s Data Centers

[Rulebreakers’ Revolutions] How SK hynix’s Design Innovations Pushed GDDR7 to New Limits of Speed

The post [Rulebreakers’ Revolutions] How CXL Tech Expands Data Center Memory Scaling Boundaries in the AI Era first appeared on SK hynix Newsroom.

SK hynix’s booth at CXL DevCon 2024

SK hynix participated in the first-ever Compute Express Link Consortium Developers Conference (CXL DevCon) held in Santa Clara, California from April 30–May 1. Organized by a group of more than 240 global semiconductor companies known as the CXL Consortium, CXL DevCon 2024 welcomed a majority of the consortium’s members to showcase their latest technologies and research results.

CXL is a technology that unifies the interfaces of different devices in a system such as semiconductor memory, storage, and logic chips. As it can increase system bandwidth and processing capacity, CXL is receiving attention as a key technology for the AI era in which high performance and capacity are essential.

Under the slogan “Memory, The Power of AI,” SK hynix showcased a range of CXL products at the conference that are set to strengthen the company’s leadership in AI memory technology.

CMM-DDR5 increases system bandwidth and capacity

During a performance demonstration, CXL Memory Module-Double Data Rate 5 (CMM-DDR5¹) expanded system bandwidth by up to 50% and capacity by up to 100% compared to systems equipped with only DDR5 DRAM. Additionally, SK hynix emphasized the benefits of a software that supports CMM-DDR5 called the Heterogeneous Memory Software Development Kit (HMSDK)². When equipped in systems with both CMM-DDR5 and standard DRAM modules, HMSDK can significantly enhance a system’s capability by relocating data to the appropriate memory device based on frequency of use.

¹Double Data Rate 5 (DDR5): A server DRAM that effectively handles the increasing demands of larger and more complex data workloads by offering enhanced bandwidth and power efficiency compared to the previous generation, DDR4.
²Heterogeneous Memory Software Development Kit (HMSDK): A software development kit specially designed to support CXL memory, a next-generation memory system based on the CXL open industry standard.

Niagara 2.0 allows multiple hosts to share capacity

SK hynix also displayed Niagara 2.0, a solution that connects multiple CXL memories together to allow numerous hosts such as CPUs and GPUs to optimally share their capacity. This eliminates idle memory usage while reducing power consumption.

Compared with the previous generation Niagara 1.0 which only allowed systems to share capacity with one another, Niagara 2.0 also enables the sharing of data. In turn, this reduces inefficiencies such as redundant data processing and, therefore, improves overall system performance. As a result, these CXL products are expected to be used in AI and high-performance computing (HPC) systems in the future.

Wonha Choi speaking about the present and future of CXL technology

During the presentation session at the conference, SK hynix’s Distinguished Engineer Wonha Choi of the Next-Gen Memory & Storage team gave a talk titled “Enabling CXL Memory Module, Exploring Memory Expansion Use Cases & Beyond”. The presentation covered the background of CXL’s adoption through to the technology’s components, research cases and performance, and anticipated applications in the future.

Following its participation at CXL DevCon 2024, SK hynix plans to strengthen its AI memory leadership by advancing CXL technology for its expanding product lineup.

The post SK hynix Presents CXL Memory Solutions Set to Power the AI Era at CXL DevCon 2024 first appeared on SK hynix Newsroom.