The world runs on data—a ceaseless tide of ones and zeros surging through networks, powering everything from streaming services to AI. To handle this data deluge, data centers must employ more advanced memory solutions that meet ever-growing performance demands.

However, traditional methods of scaling memory are facing limitations. The constraints of processors and memory technologies, coupled with escalating costs and power consumption for data centers, have highlighted the need for a revolutionary approach. Enter Compute Express Link^® (CXL^®), a transformative memory interconnect technology designed to tackle the challenges of the AI era.

This Rulebreakers’ Revolutions  episode will cover SK hynix’s development of CXL solutions, detailing how the company overcame obstacles, such as a lack of industry specifications, to become a leader in the CXL field and a key contributor to the CXL ecosystem.

Mission: Harnessing New Interconnect Tech for Memory Scaling

In the AI era, data centers need to continuously expand their memory capacity to handle ever-growing volumes of data. However, scaling memory capacity through traditional methods is becoming prohibitively expensive and inefficient. For example, adding terabyte (TB)-scale memory to a single CPU system can significantly increase the total cost of ownership¹ (TCO) and power consumption. Attempting to address this by increasing memory channels or integrating higher capacity memory often results in even greater power usage and heat generation, further inflating cooling system and management costs. This has underlined the need for innovative memory system designs which can process data faster, more efficiently, and cost effectively.

¹Total cost of ownership: The complete cost of acquiring, operating, and maintaining an asset, including purchase, energy, and maintenance expenses.

Data center memory capacity needs to increase to handle the growing demands of the AI era

Over the past decade, the industry has been developing new memory interconnect technologies to meet this market demand. Memory interconnect technology refers to the method by which processors exchange data with memory, playing a critical role in determining the speed and efficiency of data processing. In traditional memory architectures, memory is physically connected to a nearby single processor, which can lead to an over-provision of memory resources when applications are not using the memory. New memory interconnect technologies such as CXL can overcome this issue by allowing multiple processors to share memory for improved efficiency.

This has led to great interest in CXL, however developing the technology would prove challenging as there was no precedent for the process and initially no industry-established specifications. Without the JEDEC² specifications which are generally provided for DRAM products, the development process for CXL was fundamentally more complex than usual.

²JEDEC Solid State Technology Association: With over 350 member companies, JEDEC is the global leader in developing open standards for the microelectronics industry.

Faced with having to develop new CXL products without industry specifications to break the barriers of memory scaling, SK hynix tapped into its internal expertise and collaborated with industry partners.

Into the Unknown: Developing Pioneering CXL Tech From Scratch

Following the introduction of CXL in 2019, SK hynix soon recognized the technology’s capability to meet ever-growing memory scaling needs. As an open industry-standard interconnect, CXL unifies the interfaces of different system devices such as memory, storage, and processors. It supports features such as memory sharing, allowing multiple processors to access the same memory for improved data sharing, and memory pooling, in which memory from a common pool is assigned to processors for enhanced efficiency. Furthermore, CXL also enables memory switching, allowing hundreds of devices such as processors to share memory resources while independently processing data.

In addition to these innovative features, SK hynix became further convinced of CXL’s immense potential after observing increased market and customer commitment to the technology and identifying its promise in addressing technical and cost challenges. However, the company had to begin the project by overcoming a significant obstacle—a lack of industry specifications. SK hynix therefore soon set about developing its own basic requirement document after participating in CXL standardization activities and working with customers to define specifications. The company also collaborated with CXL controller companies to define controller requirements for the specifications document. Furthermore, the company has worked with JEDEC and the CXL Consortium³ to enhance DRAM-related specifications for industry CXL standards.

³CXL Consortium: An open industry standards group that develops technical specifications for CXL.

SK hynix’s CXL technology overcomes memory scaling challenges by expanding system capacity and bandwidth

Having helped set industry standards and develop relevant specifications, the company has accelerated its CXL development. For this process, SK hynix has identified key criteria to meet customer requirements—cost efficiency, high capacity, optimized bandwidth, and reliability.

First, cost efficiency is paramount in CXL development. To counteract the high cost of CXL controllers, it is crucial to minimize the costs of memory media such as modules. As high capacity is essential to facilitate large-scale data processing, the company determined CXL memory should offer storage two to four times larger than existing DDR products. Furthermore, bandwidth design must be optimized to utilize the full performance potential of CXL modules. Finally, reliability and data integrity must match the high standards of the host memory to earn customer trust.

To meet these criteria, multiple departments across SK hynix are working on making terabyte-scale memory more affordable and efficient. This includes pioneering memory pooling technologies that enable resource sharing among multiple devices and developing NMP⁴ technologies to handle data close to its source. These innovations are poised to deliver significant benefits to applications such as high-performance computing (HPC), in-memory databases, and AI.

⁴Near-memory processing (NMP): A technique that performs computations near data storage, reducing latency and boosting performance in high-bandwidth tasks like AI and HPC.

Through these efforts, SK hynix has been able to advance the development of groundbreaking CXL products which are set to revolutionize the memory field.

SK hynix’s Growing Product Lineup Driving the Future of CXL

Since developing its first DDR5-based CXL sample in 2022, SK hynix has been strengthening its CXL portfolio which includes the innovative CXL Memory Module-Double Data Rate 5 (CMM-DDR5). Leveraging high-speed PCIe Gen5 connections, CMM-DDR5 ensures smooth and rapid data processing. Available with up to 128 GB of storage, CMM-DDR5 also offers the high capacity required for the demands of today’s AI and HPC applications. In addition, the module boasts high levels of power efficiency and security.

Real-world performance tests highlight the transformative impact of CMM-DDR5. The product can expand system bandwidth by up to 82% and capacity by up to 100% compared to systems equipped with only DDR5 DRAM. Tests also showed how AI workloads experienced a 31% increase in token per second performance and that HPC enjoyed a 33% improvement in throughput efficiency. As well as providing outstanding performance, CMM-DDR5 has aligned with both JEDEC and CXL Consortium standards. Currently, the verification and certification of CMM-DDR5 is being carried out by customers as the product moves closer to mass production.

SK hynix’s CXL-based CMM-DDR5 enhances AI and HPC performance

SK hynix’s other CXL solutions include Niagara 2.0, an integrated hardware and software solution that allows multiple hosts to efficiently share large memory pools to minimize unused or underutilized memory. Furthermore, CXL Memory Module-Ax (CMM-Ax), a high-performance memory module optimized for computational workloads, is notable for improving AI and data center efficiency.

Beyond hardware advancements, SK hynix has developed the Heterogeneous Memory Software Development Kit (HSMDK) to maximize the potential of its CXL memory. This software toolkit has even been integrated into Linux’s operating system, further enhancing its accessibility and usability. The development of both hardware and software solutions as well as its standardization efforts highlights how SK hynix is committed to creating a thriving CXL ecosystem.

Rulebreaker Interview: “Thomas” Wonha Choi, Next-Gen Memory & Storage

In an interview with the SK hynix Newsroom, Distinguished Engineer⁵ (DE) “Thomas” Wonha Choi of Next-Gen Memory & Storage discussed the company’s rulebreaking mentality for developing CXL technology. Responsible for standardization efforts with JEDEC and the CXL Consortium and pathfinding next-generation memory such as CXL, Choi spoke about CXL’s development and its future impact.

⁵Distinguished Engineer: Senior SK hynix engineers who excel in their fields and are tasked with solving technical challenges and mentoring the next generation.

<Other articles from this series>

[Rulebreakers’ Revolutions] How MR-MUF’s Heat Control Breakthrough Elevated HBM to New Heights

[Rulebreakers’ Revolutions] How SK hynix Broke Barriers in Mobile DRAM Scaling With World-First HKMG Application

[Rulebreakers’ Revolutions] Innovative Design Scheme Helps HBM3E Reach New Heights

[Rulebreakers’ Revolutions] How SK hynix’s Server DRAM Validation Process Succeeds in a Diverse Server CPU Market

[Rulebreakers’ Revolutions] Flexible & Collaborative eSSD Virtualization Development for Today’s Data Centers

[Rulebreakers’ Revolutions] How SK hynix’s Design Innovations Pushed GDDR7 to New Limits of Speed

The post [Rulebreakers’ Revolutions] How CXL Tech Expands Data Center Memory Scaling Boundaries in the AI Era first appeared on SK hynix Newsroom.

SK hynix joined the August 2-4 event at the Santa Clara Convention Center in California, the first in-person FMS since 2019. In the event’s keynote and other sessions, company executives and project leaders discussed new products and the challenges that went into their development. Among the keynote speakers were SK hynix Executive Vice President and Head of NAND Development Jungdal Choi and Sanjay S. Talreja, senior vice president and general manager at US subsidiary Solidigm.

Image 1. SK hynix’s booth at FMS 2022, where attendees could experience the company’s latest products

In their keynote titled “Unlocking the Potential of Data – the New Paradigm of Storage”, the executives explained how innovative NAND technology is addressing the ever-growing need for “DataCosm”, the unique confluence of data, storage, and computing.

Transferring More Data, Faster: The World’s Most Advanced NAND

During the August 2 presentation, Choi unveiled the world’s first 238-layer 512Gb Triple-Level Cell (TLC) 4D NAND with plans to start mass production in the first half of 2023. What makes this latest product stand out is that it is simultaneously the most layered and smallest in area, in turn dramatically improving productivity, data transfer speed, and power efficiency.

Talreja also unveiled the world’s first working Penta-Level Cell (PLC) SSD with the ability to store five bits of data per memory cell, 25% more data in the same footprint compared to quad-level cell (QLC) SSDs. The company is now well-placed to grow in areas including modern workloads such as AI, machine learning and big data analytics; building out of 5G infrastructure and displacing hard disk drives, which still store more than 85% of all data center data.

Image 2. Jungdal Choi and Sanjay S. Talreja share the stage as they deliver their keynote speech

Collaboration for Memory for CXL & Next-Generation Storage

SK hynix’s Head of DRAM Product Planning Uksong Kang and Woosuk Chung, director of Storage SA, also introduced their achievements during technical seminars. During the MemVerge session on August 2, Kang announced development of the company’s first DDR5 DRAM-based Compute Express Link (CXL) samples. The new product redefines memory-as-a-service thanks to CXL memory’s bandwidth/capacity expansion, media differentiation, controller differentiation, and pooling. Kang also stressed the importance of industry collaboration to build CXL memory ecosystem.

Image 3. CXL sample in hand, Uksong Kang introduces the product during his sponsored session

Meanwhile, during his Thursday (August 4) session, Chung demonstrated a next-generation intelligent storage product and the world’s first ordered Key Value Store Computational Storage Device (KV-CSD) that improves write and read capabilities for high performance computing. Developed in partnership with Los Alamos National Laboratory (LANL), the now two-year partnership played an important role in the development of the product, with performance testing performed on LANL’s supercomputer.

Santosh Kumar, director of NAND technical marketing at SK hynix, joined moderator Paul Saffo from Stanford University and executives from other NAND companies in a roundtable discussion “NAND Flash and its Impact on the World” on August 4. The speakers discussed the impact NAND Flash has had on the world over the past 35 years, future applications of NAND Flash and the challenges that memory technologies continue to face.

Image 4. Santosh Kumar (second from the left) participates in the “NAND Flash and its Impact on the World” roundtable discussion

After a successful FMS 2022, SK hynix’s Choi hopes to leverage the momentum to develop more products that push the technology barrier. “We will continue innovations to find breakthroughs in technological challenges,” he said.

The post SK hynix Unlocks NAND Memory Potential with Innovative Products at FMS 2022 first appeared on SK hynix Newsroom.