The world runs on data—a ceaseless tide of ones and zeros surging through networks, powering everything from streaming services to AI. To handle this data deluge, data centers must employ more advanced memory solutions that meet ever-growing performance demands.

However, traditional methods of scaling memory are facing limitations. The constraints of processors and memory technologies, coupled with escalating costs and power consumption for data centers, have highlighted the need for a revolutionary approach. Enter Compute Express Link^® (CXL^®), a transformative memory interconnect technology designed to tackle the challenges of the AI era.

This Rulebreakers’ Revolutions  episode will cover SK hynix’s development of CXL solutions, detailing how the company overcame obstacles, such as a lack of industry specifications, to become a leader in the CXL field and a key contributor to the CXL ecosystem.

Mission: Harnessing New Interconnect Tech for Memory Scaling

In the AI era, data centers need to continuously expand their memory capacity to handle ever-growing volumes of data. However, scaling memory capacity through traditional methods is becoming prohibitively expensive and inefficient. For example, adding terabyte (TB)-scale memory to a single CPU system can significantly increase the total cost of ownership¹ (TCO) and power consumption. Attempting to address this by increasing memory channels or integrating higher capacity memory often results in even greater power usage and heat generation, further inflating cooling system and management costs. This has underlined the need for innovative memory system designs which can process data faster, more efficiently, and cost effectively.

¹Total cost of ownership: The complete cost of acquiring, operating, and maintaining an asset, including purchase, energy, and maintenance expenses.

Data center memory capacity needs to increase to handle the growing demands of the AI era

Over the past decade, the industry has been developing new memory interconnect technologies to meet this market demand. Memory interconnect technology refers to the method by which processors exchange data with memory, playing a critical role in determining the speed and efficiency of data processing. In traditional memory architectures, memory is physically connected to a nearby single processor, which can lead to an over-provision of memory resources when applications are not using the memory. New memory interconnect technologies such as CXL can overcome this issue by allowing multiple processors to share memory for improved efficiency.

This has led to great interest in CXL, however developing the technology would prove challenging as there was no precedent for the process and initially no industry-established specifications. Without the JEDEC² specifications which are generally provided for DRAM products, the development process for CXL was fundamentally more complex than usual.

²JEDEC Solid State Technology Association: With over 350 member companies, JEDEC is the global leader in developing open standards for the microelectronics industry.

Faced with having to develop new CXL products without industry specifications to break the barriers of memory scaling, SK hynix tapped into its internal expertise and collaborated with industry partners.

Into the Unknown: Developing Pioneering CXL Tech From Scratch

Following the introduction of CXL in 2019, SK hynix soon recognized the technology’s capability to meet ever-growing memory scaling needs. As an open industry-standard interconnect, CXL unifies the interfaces of different system devices such as memory, storage, and processors. It supports features such as memory sharing, allowing multiple processors to access the same memory for improved data sharing, and memory pooling, in which memory from a common pool is assigned to processors for enhanced efficiency. Furthermore, CXL also enables memory switching, allowing hundreds of devices such as processors to share memory resources while independently processing data.

In addition to these innovative features, SK hynix became further convinced of CXL’s immense potential after observing increased market and customer commitment to the technology and identifying its promise in addressing technical and cost challenges. However, the company had to begin the project by overcoming a significant obstacle—a lack of industry specifications. SK hynix therefore soon set about developing its own basic requirement document after participating in CXL standardization activities and working with customers to define specifications. The company also collaborated with CXL controller companies to define controller requirements for the specifications document. Furthermore, the company has worked with JEDEC and the CXL Consortium³ to enhance DRAM-related specifications for industry CXL standards.

³CXL Consortium: An open industry standards group that develops technical specifications for CXL.

SK hynix’s CXL technology overcomes memory scaling challenges by expanding system capacity and bandwidth

Having helped set industry standards and develop relevant specifications, the company has accelerated its CXL development. For this process, SK hynix has identified key criteria to meet customer requirements—cost efficiency, high capacity, optimized bandwidth, and reliability.

First, cost efficiency is paramount in CXL development. To counteract the high cost of CXL controllers, it is crucial to minimize the costs of memory media such as modules. As high capacity is essential to facilitate large-scale data processing, the company determined CXL memory should offer storage two to four times larger than existing DDR products. Furthermore, bandwidth design must be optimized to utilize the full performance potential of CXL modules. Finally, reliability and data integrity must match the high standards of the host memory to earn customer trust.

To meet these criteria, multiple departments across SK hynix are working on making terabyte-scale memory more affordable and efficient. This includes pioneering memory pooling technologies that enable resource sharing among multiple devices and developing NMP⁴ technologies to handle data close to its source. These innovations are poised to deliver significant benefits to applications such as high-performance computing (HPC), in-memory databases, and AI.

⁴Near-memory processing (NMP): A technique that performs computations near data storage, reducing latency and boosting performance in high-bandwidth tasks like AI and HPC.

Through these efforts, SK hynix has been able to advance the development of groundbreaking CXL products which are set to revolutionize the memory field.

SK hynix’s Growing Product Lineup Driving the Future of CXL

Since developing its first DDR5-based CXL sample in 2022, SK hynix has been strengthening its CXL portfolio which includes the innovative CXL Memory Module-Double Data Rate 5 (CMM-DDR5). Leveraging high-speed PCIe Gen5 connections, CMM-DDR5 ensures smooth and rapid data processing. Available with up to 128 GB of storage, CMM-DDR5 also offers the high capacity required for the demands of today’s AI and HPC applications. In addition, the module boasts high levels of power efficiency and security.

Real-world performance tests highlight the transformative impact of CMM-DDR5. The product can expand system bandwidth by up to 82% and capacity by up to 100% compared to systems equipped with only DDR5 DRAM. Tests also showed how AI workloads experienced a 31% increase in token per second performance and that HPC enjoyed a 33% improvement in throughput efficiency. As well as providing outstanding performance, CMM-DDR5 has aligned with both JEDEC and CXL Consortium standards. Currently, the verification and certification of CMM-DDR5 is being carried out by customers as the product moves closer to mass production.

SK hynix’s CXL-based CMM-DDR5 enhances AI and HPC performance

SK hynix’s other CXL solutions include Niagara 2.0, an integrated hardware and software solution that allows multiple hosts to efficiently share large memory pools to minimize unused or underutilized memory. Furthermore, CXL Memory Module-Ax (CMM-Ax), a high-performance memory module optimized for computational workloads, is notable for improving AI and data center efficiency.

Beyond hardware advancements, SK hynix has developed the Heterogeneous Memory Software Development Kit (HSMDK) to maximize the potential of its CXL memory. This software toolkit has even been integrated into Linux’s operating system, further enhancing its accessibility and usability. The development of both hardware and software solutions as well as its standardization efforts highlights how SK hynix is committed to creating a thriving CXL ecosystem.

Rulebreaker Interview: “Thomas” Wonha Choi, Next-Gen Memory & Storage

In an interview with the SK hynix Newsroom, Distinguished Engineer⁵ (DE) “Thomas” Wonha Choi of Next-Gen Memory & Storage discussed the company’s rulebreaking mentality for developing CXL technology. Responsible for standardization efforts with JEDEC and the CXL Consortium and pathfinding next-generation memory such as CXL, Choi spoke about CXL’s development and its future impact.

⁵Distinguished Engineer: Senior SK hynix engineers who excel in their fields and are tasked with solving technical challenges and mentoring the next generation.

<Other articles from this series>

[Rulebreakers’ Revolutions] How MR-MUF’s Heat Control Breakthrough Elevated HBM to New Heights

[Rulebreakers’ Revolutions] How SK hynix Broke Barriers in Mobile DRAM Scaling With World-First HKMG Application

[Rulebreakers’ Revolutions] Innovative Design Scheme Helps HBM3E Reach New Heights

[Rulebreakers’ Revolutions] How SK hynix’s Server DRAM Validation Process Succeeds in a Diverse Server CPU Market

[Rulebreakers’ Revolutions] Flexible & Collaborative eSSD Virtualization Development for Today’s Data Centers

[Rulebreakers’ Revolutions] How SK hynix’s Design Innovations Pushed GDDR7 to New Limits of Speed

The post [Rulebreakers’ Revolutions] How CXL Tech Expands Data Center Memory Scaling Boundaries in the AI Era first appeared on SK hynix Newsroom.

• Development of 61TB product based on QLC technology, strengthening synergy with Solidigm in the AI data center SSD market
• Data transfer speed of up to 32GT/s with PCIe 5.0, with doubled sequential read performance compared to PCIe 4.0 products
• SK hynix to build a foundation for growth to become a full-stack AI memory provider with leadership in high-capacity eSSD technology

Seoul, December 18, 2024

SK hynix Inc. (or “the company”, www.skhynix.com) announced today that it has completed development of its high-capacity SSD product, ‘PS1012 U.2¹’, designed for AI data centers.

¹U.2: A type of Form Factor for SSDs, typically 2.5 inches in size, primarily used in servers or high-performance workstations. It is known for large capacity storage and high durability.

As the era of AI accelerates, the demand for high-performance enterprise SSDs (eSSD) is rapidly increasing, and QLC² technology, which enables high capacity, has become the industry standard. In line with this trend, SK hynix has developed a 61TB product using this technology and introduced it to the market.

²QLC: NAND flash is divided into SLC (Single Level Cell), MLC (Multi Level Cell), TLC (Triple Level Cell), QLC (Quadruple Level Cell), and PLC (Penta Level Cell) depending on how much information is stored in one cell. As the amount of information stored increases, more data can be stored in the same area.

SK hynix has been leading the SSD market for AI data centers with Solidigm, a subsidiary which commercialized QLC-based eSSD for the first time in the world. With the development of PS1012, the company expects to build a balanced SSD portfolio, thereby maximizing synergy between the two companies.

With the latest 5th generation (Gen5) PCIe³, PS1012 doubles its bandwidth compared to 4th generation based products. As a result, the data transfer speed reaches 32GT/s (Giga-transfers per second)⁴, with the sequential read performance of 13GB/s (Gigabyte per second), which is twice that of previous generation products.

³Peripheral Component Interconnect express (PCIe): A high-speed input/output interface with a serial structure used on the main board of a digital device.
⁴Giga-transfers per second (GT/s): Number of operations or information transferred per second

In addition, the company developed this product to support the OCP 2.0 version, enhancing its compatibility with various data center server devices of global AI customers.

⁵Open Compute Project (OCP): An international consultative body that involves major data center companies around the world to discuss hardware, software and eSSD standards for building ultra-high-efficiency data centers.

SK hynix plans to supply the sample of the new product to global server manufacturers within this year for product evaluation, and based on this, it plans to expand its product line to 122TB in the third quarter of next year. The company also aims to lead the SSD market for ultra-high capacity data centers by developing 244TB products based on the world’s highest 321-high 4D NAND developed in November, to overcome the capacity limitations of eSSD.

“SK hynix and Solidigm are strengthening our QLC-based high-capacity SSD lineup to solidify our technological leadership in NAND solutions for AI,” said Ahn Hyun, President and Chief Development Officer of SK hynix. “In the future, we will lay the foundation for growth to become a Full Stack AI memory provider by meeting the diverse needs of AI data center customers based on our high competitiveness in the eSSD field.”

About SK hynix Inc.

SK hynix Inc., headquartered in Korea, is the world’s top tier semiconductor supplier offering Dynamic Random Access Memory chips (“DRAM”), flash memory chips (“NAND flash”) and CMOS Image Sensors (“CIS”) for a wide range of distinguished customers globally. The Company’s shares are traded on the Korea Exchange, and the Global Depository shares are listed on the Luxembourg Stock Exchange. Further information about SK hynix is available at www.skhynix.com, news.skhynix.com.

Media Contact

SK hynix Inc.
Global Public Relations

Technical Leader
Sooyeon Lee
E-Mail: global_newsroom@skhynix.com

Technical Leader
Kanga Kong
E-Mail: global_newsroom@skhynix.com

The post SK hynix Develops ‘PS1012 U.2’, High Capacity SSD for AI Data Centers first appeared on SK hynix Newsroom.

SK hynix is showcasing its leading AI and data center memory products at the 2024 Open Compute Project (OCP) Global Summit held October 15–17 in San Jose, California. The annual summit brings together industry leaders to discuss advancements in open source hardware and data center technologies. This year, the event’s theme is “From Ideas to Impact,” which aims to foster the realization of theoretical concepts into real-world technologies.

SK hynix’s booth at the 2024 OCP Global Summit

In addition to presenting its advanced memory products at the summit, SK hynix is also strengthening key industry partnerships and sharing its AI memory expertise through insightful presentations. This year, the company is holding eight sessions—up from five in 2023—on topics including HBM¹ and CMS².

¹High Bandwidth Memory (HBM): A high-value, high-performance product that revolutionizes data processing speeds by connecting multiple DRAM chips with through-silicon via (TSV).
²Compute Memory Solution (CMS): A memory solution optimized for AI, HPC, and data centers, enhancing speed, efficiency, and scalability for compute-heavy workloads.

Displays & Demonstrations at the Booth: Transforming AI & Data Centers

Visitors to SK hynix’s booth can see a range of the company’s cutting-edge AI and data center solutions and view demonstrations with high-performance customer systems.

SK hynix’s groundbreaking AI and data center products on display

Among the products being demonstrated is CMM-Ax³, formerly known as CMS 2.0, which was shown under its new name for the first time. For the demonstration, SK hynix is highlighting the product’s role in next-generation compute memory for AI infrastructure, particularly for multi-modal applications. Meanwhile, a heterogeneous memory management solution involving CMM (CXL Memory Module)-DDR5⁴ and HMSDK⁵ is also being demonstrated as well as Niagara 2.0⁶, which is illustrating its efficient utilization of pooled memory resources. Another demonstration is highlighting the computational storage drive’s (CSD) AI storage capabilities for large-scale training systems.

³CXL Memory Module-Ax (CMM-Ax): A high-performance memory module optimized for computational workloads, improving AI and data center efficiency.
⁴CXL Memory Module-DDR5 (CMM-DDR5): A next-gen DDR5 memory module based on CXL that enhances system bandwidth, speed, and performance for AI, cloud, and high-performance computing.
⁵Heterogeneous Memory Software Development Kit (HMSDK): A software development kit specially designed to support CXL memory, a next-generation memory system based on the CXL open industry standard.
⁶Niagara 2.0: An integrated HW/SW solution for pooled memory that allows multiple hosts (CPUs and GPUs) to efficiently share large memory pools to minimize unused or underutilized memory known as stranded memory. By supporting optimal data placement through hot and cold data detection, it can significantly improve system performance.

SK hynix is showcasing CXL-based solutions including CMM-Ax and Niagara 2.0

In addition, a live demonstration of the GDDR6-AiM⁷-based accelerator card AiMX is being held using Meta’s latest large language model (LLM), Llama3 70B, which has 70 billion parameters. The demonstration shows AiMX’s capability to tackle industry challenges. For example, data centers’ LLM services improve GPU efficiency by simultaneously processing requests from multiple users. However, as the length of the generated token increases, the computation of the attention layer⁸ increases, lowering GPU efficiency. Through the demonstration, AiMX shows it can overcome this issue by handling large amounts of data while offering greater efficiency and lower power consumption compared to the latest accelerators.

⁷Accelerator-in-Memory (AiM): SK hynix’s PIM semiconductor product. PIM is next-generation technology that adds computational functions to memory semiconductors, solving the data transfer bottleneck in AI and big data processing fields.
⁸Attention layer: A mechanism which allows a model to determine the importance of input data to focus on more relevant information.

The AiMX card is being demonstrated with Meta’s latest LLM, Llama3 70B

The company also displayed a range of its industry-leading AI memory and data center products. HBM3E is shown along with the NVIDIA H200 Tensor Core GPU and NVIDIA GB200 Grace Blackwell Superchip. The booth also features SK hynix’s DDR5 RDIMM and MCR DIMM server DRAM, as well as enterprise SSDs (eSSDs), with Supermicro servers. The DDR5 products on display include the world’s first DDR5 DRAM built using the 1c node, the sixth generation of the 10nm process technology. Designed to meet the growing computational and energy demands of AI-driven data centers, the 16 Gb 1c DDR5 offers improved performance and energy efficiency from the previous generation. Additionally, SK hynix unveiled the 1bnm 96 GB DDR5 which can reach speeds of up to 7,200 megabits per second (Mbps), making it ideal for supporting massive data flows in data centers.

HBM3E is presented alongside NVIDIA’s H200 and GB200, while DDR5 RDIMM and MCR DIMM products are shown with Supermicro servers

In the SSD section, the Gen5 eSSDs PS1010 and PS1030 and the Gen4 PE9010 are among those on display. Offering ultra-fast read/write speeds, these SSD solutions are vital for accelerating AI training and inference in large-scale environments. Through all these innovations, SK hynix is continuing to lead the way in AI memory and storage solutions, fueling the future of AI and transforming data center operations.

Gen5 eSSDs showcased at the summit

Expanded Presentation Sessions: Sharing AI Memory Expertise

Reflecting SK hynix’s growing influence as the leading AI memory provider, the company is holding eight insightful sessions—three more than in 2023—on the future of AI memory solutions.

SK hynix is sharing its industry expertise through eight presentations

• Youngpyo Joo, head of Software Solution: Joo discussed how AI-driven technological advancements are reshaping computing architecture, focusing on innovations such as CXL^®9-based memory and computational memory solutions for AI workloads.
• Technical Leader Jungmin Choi, Composable System: In his talk, Choi explored how memory disaggregation can improve system efficiency.
• Technical Leader Honggyu Kim, System SW: Kim explained how SK hynix’s HMSDK solution optimizes memory pooling and sharing in AI workloads, improving performance and reducing network congestion.
• Technical Leader Younsoo Kim, DRAM Technology Planning: Kim addressed the future of HBM memory and its pivotal role in AI applications in his presentation.
• Euicheol Lim, head of Solution Advanced Technology (Solution AT): Lim discussed the role of SK hynix’s AiMX card to reduce operational costs in LLM processing in data centers and on-device services.
• Technical Leader Kevin Tang and Team Leader Jongryool Kim, AI System Infra: Tang and Kim delivered a joint presentation about improving LLM training efficiency and scalability through checkpoint¹⁰
• Hoshik Kim, head of SOLAB: As part of a panel discussion, Kim discussed various issues, solutions, and visions that near-data computing¹¹ needs to address for application to various real systems.
• Technical Leader Myoungseo Kim, AI Open Innovation: Kim will deliver the final talk with Vikrant Soman, Solution Architect at Uber, and Jackrabbit Labs CTO Grant Mackey on CMS composable memory architecture and the need for scalable, interoperable memory solutions. The presentation will reveal the results of SK hynix’s CXL pooled memory prototype integration with Jackrabbit Labs’ open source cluster orchestration software to address the pain point of stranded memory in the Uber Data Center Kubernetes Cluster environment.

⁹Compute Express Link (CXL): A PCIe-based next-generation interconnect protocol on which high-performance computing systems are based.
¹⁰Checkpoint: A technology that stores model parameters and related key data at a specific point during the learning (training) process, enabling the learning process to restart from the saved point in case of a system failure.
¹¹Near-data computing: A computing method aimed at addressing the bottleneck issue between the memory and processor, a limitation of the von Neumann architecture. It delivers only refined data processed by memory to the processor to minimize the movement of data within the system, improving performance and cost efficiency.

Shaping the Future: Advancing AI Memory & Data Center Innovation

At the 2024 OCP Global Summit, SK hynix is reinforcing its industry leadership by not only showcasing its advanced AI memory and data center solutions but also sharing its expertise. Looking ahead, the company is dedicated to pioneering breakthroughs that will define the next generation of AI and data center technologies.

The post SK hynix at the 2024 OCP Global Summit: Leading the Future of AI & Data Center Memory Solutions first appeared on SK hynix Newsroom.

• New SSD delivers 2x performance, over 30% greater power efficiency than previous generation by applying 5th-generation PCIe
• Mass production to begin in 2Q next year following customer qualification
• SK hynix to enhance data center SSD portfolio to meet diverse customer needs
• Company to consolidate position as No. 1 global AI memory provider in NAND solutions such as SSDs following success with HBM

Seoul, September 11, 2024

SK hynix Inc. (or “the company”, www.skhynix.com) announced today that it has developed PEB110 E1.S (PEB110), a high-performance solid-state drive (SSD) for data centers.

With the advent of the AI era, customer demand for high-performance NAND solutions such as SSDs for data centers, as well as ultra-fast DRAM chips including high bandwidth memory (HBM), is growing. In line with this trend, the company has developed and introduced a new product with improved data processing speed and power efficiency by applying the fifth-generation (Gen5) PCIe¹ specifications.

SK hynix expects to meet diverse customer needs with a more robust SSD portfolio following successful mass production of PS1010² with the introduction of PEB110.

¹Peripheral Component Interconnect Express (PCle): A serial-structured, high-speed input/output interface used on the motherboard of digital devices.
²PS1010: A data center/server-oriented SSD with ultra-high performance and high capacity that is based on PCIe Gen5 E3.S and U.2/3.

The company is currently in the qualification process with a global data center customer and plans to begin mass production of the product in the second quarter of next year, pending qualification.

PCle Gen5, applied to the new product, provides twice the bandwidth of the fourth generation (Gen4), enabling PEB110 to achieve data transfer rates of up to 32 gigatransfers per second (GT/s). This enables PEB110 to double the performance of the previous generation and improve power efficiency by more than 30%.

SK hynix has also applied the security protocol and data model technology, or SPDM, to PEB110, for the first time for its data-center SSDs to significantly enhance information security features.

SPDM, a key security solution specialized in protecting server systems, offers secure authentication and monitoring of servers. With the recent increase in cyberattacks on data centers, the company expects that PEB110 with SPDM will meet the high information security needs of customers.

The product will be released in three capacity versions—2 terabyte (TB), 4 TB, and 8 TB—and supports the OCP³ version 2.5 specifications for greater compatibility across global data centers.

³Open Compute Project (OCP): An international council of leading data center companies from around the world discussing standards for hardware, software, and eSSDs to build ultra-efficient data centers.

“The new product builds on the company’s best-in-class 238-high 4D NAND, boasting the most competitive standards in the industry in terms of cost, performance and quality,” said Ahn Hyun, Head of the N-S Committee at SK hynix.

“Looking ahead, we are on track to proceed with customer qualification and volume production to solidify our position as the No. 1 global AI memory provider in the ever-growing SSD market for data centers,” said Ahn.

About SK hynix Inc.

SK hynix Inc., headquartered in Korea, is the world’s top-tier semiconductor supplier offering Dynamic Random Access Memory chips (“DRAM”), flash memory chips (“NAND flash”), and CMOS Image Sensors (“CIS”) for a wide range of distinguished customers globally. The Company’s shares are traded on the Korea Exchange, and the Global Depository shares are listed on the Luxembourg Stock Exchange. Further information about SK hynix is available at www.skhynix.com, news.skhynix.com.

Media Contact

SK hynix Inc.
Global Public Relations

Technical Leader
Kanga Kong
E-Mail: global_newsroom@skhynix.com

Technical Leader
Sooyeon Lee
E-Mail: global_newsroom@skhynix.com

The post SK hynix Develops PEB110 E1.S for Data Centers first appeared on SK hynix Newsroom.

• SK hynix and Intel co-published performance verification white paper on DDR5’s application to Intel CPUs for use in servers
• SK hynix’s DDR5 boosts server bandwidth by 70% while lowering power consumption by 14.4% compared to the product’s previous generation
• SK hynix plans to accelerate improvement in its second-half performance with its quality-certified DDR5

SK hynix announced on September 14th that it has co-published a white paper with Intel which reveals that its DDR5 server DRAM applied to Intel’s CPUs demonstrated industry-leading performance levels. This white paper was released simultaneously on the SK hynix and Intel websites.

The two companies have worked closely together since the beginning of DDR5’s development, and this white paper shows the results of performance evaluations of SK hynix’s DDR5 when applied to 4th Gen Intel^® Xeon^® Scalable processors¹ (hereinafter referred to as Xeon) that took place over the past eight months.

Released at a time when the server industry has called for low-power, high-performance semiconductors, the white paper highlights that SK hynix and Intel will usher in an era of more advanced data centers through memory and CPUs that deliver industry-leading performance and energy efficiency.

[Read the full white paper]

The white paper reveals that SK hynix’s DDR5 uses 14.4% less power than DDR4, while the 4th Gen Intel^® Xeon^® Scalable processors offer performance efficiency that is 2.9 times greater than the previous generation². In servers with Xeon, DDR5 achieved an improved performance per watt³ that was 1.22 times higher for integer computations and 1.11 times higher for floating point⁴ computations compared to DDR4.

Accordingly, the two companies expect the energy efficiency provided by DDR5 and Xeon to allow server chip customers to build more sustainable data centers. The operation of such cost-effective data centers is also expected to help customers save on the total cost of ownership (TCO)⁵.

“As revealed in the white paper, servers equipped with SK hynix’s DDR5 and Intel’s CPUs enable faster data processing speeds while consuming less power compared to the products’ previous generations. In particular, they can efficiently utilize high-density DRAMs required for processing vast amounts of data in applications such as generative AI,” said Sungsoo Ryu, head of DRAM Product Planning at SK hynix.

“We hope that our server chip customers will use the valuable information included in this white paper to drive their business forward,” he added.

Dr. Dimitrios Ziakas, Intel’s vice president of Memory and IO Technologies, said: “Intel has been collaborating with SK hynix and memory industry partners to enable high-performance DDR5 DRAM with 4th Gen Intel^® Xeon^® Scalable processors. These efforts ensure that we provide robust high-performance and energy-efficient data center system solutions to benefit our mutual customers.”

SK hynix will strengthen its offerings in the server market through collaborations, such as this latest project with Intel. By focusing on 1anm and 1bnm DDR5, the fourth and fifth generation of the 10nm process technology, respectively, SK hynix looks to increase its financial performance and presence in the server market as demand for server DRAMs is expected to rise in the second half of 2023.

[At a Glance] The Intel and SK hynix DDR5 Ecosystem White Paper (Read the full white paper)

• Bandwidth 70%↑⁶, Power Consumption 14.4%↓⁷, Integer Computations 1.59 times↑

The white paper details test data that server customers can use to guide their application of DDR5 in their servers. Highlights include the speed, performance, and power consumption that is attainable when combining DDR5 memory and Xeon.

Figure 1. Comparing the Server Bandwidth of SK hynix’s DDR4 and DDR5

Firstly, server bandwidth⁸ at the same operating speeds of 3,200 megabits per second (Mbps) increased by 20% with DDR5 compared with the previous generation, DDR4. Additionally, the server bandwidth at DDR5’s operating speed of 4,800 Mbps was 70% greater than it was at DDR4’s highest operating speed of 3,200 Mbps⁹. The overall expansion in the server bandwidth is due to the DDR5’s design improvements that minimized internal transfer latency of data and the securing of higher transfer speeds compared to DDR4.

Figure 2. Comparing Operating Speed and Power Consumption of SK hynix’s DDR4 and DDR5

Through the study, SK hynix was also able to confirm that DDR5’s power consumption was 14.4% less than DDR4 due in large part to new technologies such as High-K Metal Gate (HKMG). HKMG is a next-generation process that uses a High-K insulating film inside a DRAM transistor to prevent leakage current and improve its capacitance—the ability to store charge. Semiconductor memory products with HKMG therefore have a higher power efficiency.

Xeon’s built-in accelerators have also shown positive results in tests. Organizations can improve the average performance per watt efficiency for target workloads by up to 2.9 times utilizing the built-in accelerators compared to the previous generation of processors. The combination of these technologies enabled DDR5 and Xeon to achieve an excellent 50% increase in bandwidth and a 14.4% reduction in power usage.

The white paper also shows the computational performance of a system that combines DDR5 and Xeon. The companies used SPEC CPU 2017, a specialized benchmarking program, to compare performances.

Results from the comparison showed that integer computations were 1.59 times faster and floating point computations 1.43 times faster than the previous generation’s system. The performance per watt also saw its efficiency rise 1.22 times for integer computations and 1.11 times for floating point computations compared to the system’s predecessor.

The advanced system also performed well in the Intel Memory Latency Checker (MLC) test, a program that measures memory latency and speed. Compared to the previous generation, the read speed increased by 1.4 times and the read/write speed rose by 1.51 times.

Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries.

¹4th Gen Intel^® Xeon^® Scalable processors: These processors are Intel’s next generation of server CPUs that support PCIe Gen5 and next-generation DDR5 DRAM.

²2.9x average performance per watt efficiency improvement for targeted workloads utilizing built-in accelerators compared to the previous generation.
New Configuration: 1-node, 2x pre-production 4th Gen Intel^® Xeon^® Scalable Processor (60 cores) with integrated Intel In-Memory Analytics Accelerator (Intel IAA), on pre-production Intel platform and software, HT On, Turbo On, Total Memory 1024GB (16x64GB DDR5 4800), microcode 0x2b0000a1, 1×3.84TB P5510 NVMe, Intel^® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, QPL v0.2.1,accel-config-v3.4.6.4, ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel November 2022. Baseline: 1-node, 2x production 3rd Gen Intel^® Xeon^® Scalable Processors (40 cores) on SuperMicro SYS-220U-TNR, HT On, Turbo On, SNC Off, Total Memory 1024GB (16x64GB DDR4 3200), microcode 0xd000375, 1×3.84TB P5510 NVMe, Intel^® Ethernet Controller X540-AT2, Ubuntu 22.04.1 LTS, 5.18.12-051812-generic, ZSTD v1.5.2, RocksDB v6.4.6 (db_​bench), tested by Intel November 2022.

³Performance per watt: An indicator of how much computation is performed per watt of power consumed.

⁴Floating point: A concept about real numbers where integers and decimals are represented separately by changing the position of the decimal point to facilitate computation, as opposed to a fixed point where the decimal point is fixed in its original position. For example, if the original real number is 123.485, it can be written in floating points as 1.23485X 10^2, or 0.00123485X10^5.

⁵Total cost of ownership (TCO): The total costs of an asset that includes such as the initial investment, power, facility operations, and maintenance.

⁶Result obtained during simulations by SK hynix performed in December 2022.

⁷Power consumption comparison of DDR4 and DDR5 based on simulations by SK hynix’s power calculator.

⁸Server Bandwidth: The pathway through which data travels. An increase in bandwidth enables more data to be processed more efficiently at once.

⁹Bandwidth comparison based on increase from DDR4’s 3,200 Mbps operating speed to DDR5’s 3,200 Mbps and 4,800 Mbps speeds.

The post SK hynix’s DDR5 Key to Enabling Industry-Leading Performing Data Centers, White Paper Reveals first appeared on SK hynix Newsroom.

¹Photonics: The study of matters related to photons, which are the fundamental units of light.

Figure 1. Conceptual diagram of silicon photonics technology (Image credit: PhotonHub)

In this latest article in our series by faculty from South Korean university DGIST, Professor Sangyoon Han of the Robotics and Mechatronics Engineering department explains silicon photonics, its use in data centers and optical links, the manufacturing process of silicon photonics chips, and the technology’s future applications.

The Significance of Silicon Photonics in Data Centers

Although data centers and their significant role in operating a host of technologies may be a familiar concept to many people, it is less widely known that silicon photonics technology is essential to the running of these centers. Tens of thousands of servers in a data center are connected not by wires but by optical fibers that act as pathways for light, enabling high-bandwidth communication. Thus, the electrical signals generated in the servers need to be converted into light, or the conversion needs to happen the other way around in order for light and electrical signals to pass through the optical fibers. To facilitate this, optical transceivers—devices developed using silicon photonics technology—are applied.

Figure 2. A prototype of a silicon photonics optical transceiver (Image credit: IMEC)

Able to handle both light and electrical signals simultaneously on a single chip, silicon photonics technology is considered an ideal platform to form optical transceivers. Due to light’s low attenuation² and their parallel processing capability, optical transceivers enable communication between servers at speeds of up to 400 Gbps (gigabits per second). Considering that electrical wires can only achieve speeds of single-digit Gbps, it is clear that optical transceivers are essential for the efficient and speedy operation of data centers. Subsequently, around a million optical transceivers are used in a single large-scale data center.

²Attenuation: In the optical field, it is the rate at which the signal light decreases in intensity.

Use of Silicon Photonics in Optical Links

The application of silicon photonics spreads beyond data centers as it is currently being used in other areas which require high-bandwidth data transmission. Recently, there has been a growing trend in using optical waveguides instead of copper wires to connect chips that are located within a few centimeters of each other. This is where silicon photonics play a key role as there is active research into the use of silicon photonics-based optical links³ for interconnection between GPUs and between the cores in a multi-core CPU. These optical links are also being considered to connect CPUs and memory chips. As a prime example of such applications, Silicon Valley chip startup Ayar Labs is working with NVIDIA to develop technology that could embed optical links into a large-scale GPU system as shown in Figure 3.

³Optical link: An optical transmission channel designed to connect two end terminals or to be connected in series with other channels.

Figure 3. Example of applying optical links in chip-to-chip technology (Image credit: Ayar Labs)

How Silicon Photonics Chips Work

So, how exactly does a silicon photonics chip work? To make it easier to understand, it is helpful to compare the technology with electronic circuits. Although electronic circuits look complex, they are mostly made up of transistors and copper wires that connect the transistors. The functions of transistors and wires in an electronic circuit are parallel to the roles of optical modulators and waveguides in a silicon photonics chip, while the power supply of an electronic circuit corresponds to the role of a laser for such chips. Additionally, a photodetector is used to convert optical signals into electrical signals in a silicon photonics chip.

Figure 4. Conceptual diagram of the process of an optical modulator

Just as wires carry electrons, waveguides carry photons, or light. They work as channels that transmit light without any loss, similar to optical fibers. For silicon photonics chips, they use ultra-thin waveguides of less than 1 micrometer⁴(μm) in diameter that are manufactured using semiconductor processes. Furthermore, optical modulators alter the transmissivity of the waveguides to change the intensity of the light passing through the waveguides—this generates optical signals as shown in Figure 4.

⁴Micrometer (μm): 1 micrometer is one-millionth of a meter.

The Manufacturing Process of Silicon Photonics Chips

Figure 5. Silicon photonics chips fabricated on a 12-inch wafer (Image source: Nature)

The process of manufacturing silicon photonics chips is very similar to making electronic circuits through the CMOS process. Both processes use silicon as a base material, and for silicon photonics chips it is advantageous to be manufactured through the CMOS process rather than developing a whole new manufacturing process in terms of time, cost, and efficiency. In turn, the similarity of the processes makes it straightforward to produce silicon photonics chips on existing semiconductor production lines.

Moreover, the size of silicon photonics devices, which measure only a few micrometers, is optimal for production as they can be easily manufactured using nanometer⁵ processes. As a result, despite the relatively short history of silicon photonics, major foundries around the world have started to produce silicon photonics chips on 12-inch wafers. With more of these world-renowned foundries entering the silicon photonics business, it is foreseeable that the silicon photonics market will grow significantly going forward.

⁵Nanometer (nm): 1 nanometer is one-billionth of a meter. Therefore, 1μm = 1,000 nm.

Silicon Photonics in Autonomous Driving Sensors

Silicon photonics has not only made a significant impact in data transmission—an area where light holds an advantage—but is also being applied to a growing number of fields these days. For example, the technology is being utilized to achieve higher performance and miniaturization of autonomous driving sensors such as Light Detection and Ranging (LiDAR). Most LiDAR systems available in the market today are difficult to mass-produce at low costs because they are made by manually assembling components such as motors and lenses. But silicon photonics technology is expected to become a new solution to this as it enables these LiDAR systems to be manufactured with improved performance, energy efficiency, and lower costs. In practice, this technology will ultimately lead to a drastic fall in costs when adding an autonomous driving system to a car.

Future Applications of Silicon Photonics

Silicon photonics is also making it possible to develop new computing technologies beyond existing paradigms. Such next-generation computing technologies include: AI processors that use light’s ability of parallel processing to compute multiple AI inferences with a single physical device; quantum computing that transcends the limits of classical physics; and quantum cryptography communications that are physically impossible to wiretap.

With the emergence of advanced technologies such as large-scale AI models, the demand for improved computing power and data processing capabilities from hardware is greater than ever. As a result, the existing paradigm of electronic semiconductors will need to evolve to keep up with this strong demand, and this evolution can be materialized with silicon photonics technology. As the technology harnesses the power of light in semiconductors, physical limitations can ultimately be overcome so semiconductors can realize previously unobtainable applications. Looking ahead, it is also anticipated that the use of silicon photonics will result in colossal progress in computing and AI applications.

<Other articles from this series>

[DGIST Series] How the Quest for AI Led to Next-Generation Memory & Computing Processors

[DGIST Series] How Broadband Interface Circuits Are Evolving for Optimal Data Transfer

[DGIST Series] The Role of Semiconductor Technologies in Future Robotics

[DGIST Series] The Technologies Handling the Growing Data Demands in Healthcare

[DGIST Series] AI-Powered Micro/Nanorobots to Revolutionize Medical Field

[DGIST Series] Sensor Interfaces and ADC Circuits: Bridging the Physical and Digital Worlds

The post [DGIST Series] Silicon Photonics: Revolutionizing Data Transfers by Unleashing the Power of Light first appeared on SK hynix Newsroom.

Image Download

A generation ago, most companies dedicated space in their buildings to house the many racks of servers needed to power their internal networks. For most, this dedicated room created a double-edged sword of challenges. As the company’s networking demands grew, so did the number of servers, as well as the amount of heat generated by the machinery. To offset that heat, companies incorporated cooling systems, which in turn led to greater energy consumption and costs.

Today, many of those same companies have replaced their internal server rooms with a cloud-based service, often provided by one of the big tech companies, such as Amazon, Google or Microsoft. In theory, this shift placed the energy challenges on the tech giants who have shown that they not only compete on their service offerings but also how they do so in a more sustainable – and creative – way.

Microsoft last year made headlines with a multi-year energy-reduction experiment that involved submerging a shipping-container-sized datacenter into the waters off the coast of Scotland and tracking its performance over a two-year period. The company concluded that the concept was not only feasible but also “logistically, environmentally and economically practical.”

Technology leaders around the globe, especially those who enable data-hungry applications and platforms, are making ambitious decarbonization commitments. Google, for example, has set a 2030 deadline for around-the-clock power to all of its offices and datacenters from clean sources. Last year, the company consumed 67 percent of its datacenter electricity from renewable sources, with some individual datacenters at 90 percent clean energy consumption.

In China, the Baidu Cloud Computing (Yangquan) Center was the first data center in that country to apply solar energy, wide voltage, direct grid connection technology. The center introduced a new era of green energy conservation in data centers, generating about 120,000 kWh annually while reducing carbon dioxide emission by about 110 tons per year.

In Europe, Google and Amazon Web Services (AWS) were among the initial signers of the European-based Climate Neutral Data Centre Pact, which aims for climate neutral data centers across that continent by 2030.

So far, these and other efforts seemed to have had an impact. The migration to large cloud- and hyperscale-class data centers has helped cut the global IT carbon footprint by as much as 38 percent in 2020 and, between 2010 and 2018, data center output jumped 6-fold but data center energy consumption rose by only 6 percent.

Figure 1. Tech groups are the biggest corporate buyers of green energy

Image Download

But the demand for data continues to grow exponentially. Alongside the corporate IT shift came a consumer-centric leap in data via their own targeted cloud-based services, including social media. The The Financial Times reported earlier this year that the combined power usage of Amazon, Google, Microsoft, Facebook and Apple is already more than 45 terawatt-hours a year, about as much as New Zealand.

Now, the adoption of data-heavy artificial intelligence and machine learning applications by companies of all sizes is gaining traction, driving even more data processing through the networks. The forecast for growth in AI and ML adoption suggests that the demands on data centers could eventually outpace any gains in efficiency.
Already, the tech companies have become some of the largest corporate buyers of clean energy for their massive data centers – but it isn’t enough. The industry is facing increased pressure – from environmentalists to politicians to their own employees – to do a better job when it comes to “going green.”

Figure 2. Social Value – When Replacing HDDs in Data Centers around the World with SSDs

Image Download

In a March 2021 keynote speech, SK hynix CEO Seok-hee Lee called on the semiconductor industry to accelerate innovation that addresses climate issues. For example, if data centers replaced their current hard disk drive (HDD)-based data storage systems with low-power PLC/QLC-based solid-state drives (SSDs), greenhouse gas emissions would be reduced by about 41 million tons.

The industry must look beyond the needs of today and develop technologies that can be expanded and leveraged in the future to accommodate the still-unknown demands of the future. Already, SK hynix is developing specialized semiconductor technologies that focus on a single characteristic, such as High-Bandwidth Memory (HBM) for high-speed processing and Ultra-low Power Memory (ULM) for consuming less power. The company has also declared that it will replace all the power required in the semiconductor manufacturing process with renewable energy by 2050.

Beyond that, there must also be a commitment to collaboration and cooperation among industries to address the growth of data-powered world and its impact on the environment, from academia to manufacturing. Along with SK Group’s other subsidiary companies, SK hynix has joined the RE100, a global corporate renewable energy initiative.

The post Beyond Clean Energy: Technology Must Help Address Data Center Challenges first appeared on SK hynix Newsroom.

One of the biggest trends is the expansion of health and fitness companies entering the technology space. U.S. fitness equipment manufacturer, Peloton, has been getting increased attention with its business model that combines fitness equipment and video streaming services. The company has its instructors and coaches streaming in real-time in which users can participate through the attached screen on their Peloton treadmill or stationary bike.

Source: Peloton Press Site

Image Download

It has not only the function to live-stream classes but also is able to provide a customized health service by collecting and analyzing the personal workout frequency, intensity and body mass index. All of these features have contributed to Peloton’s success in recent months and propelled it to become a household name as a disruptor in the fitness industry.

Source: Mirror

Image Download

Other companies like Mirror have also benefited from the success of this technology in the space. The company provides customers with an interactive mirror-like screen that tracks and teaches you as you work out, using artificial reality (AR) technology to make it feel like your personal trainer is in the room with you. The device comes with a Bluetooth heart rate monitor so you can truly track how hard you’re working and allows you to compete with others in live and on-demand classes.

As these new business models look to add more technology to interact more intimately with consumers, they are generating an overwhelming volume of data with high-resolution real-time video data from live streamed fitness classes, and the collection of user information such as workout frequency, intensity, and body mass index. According to Netflix.com, streaming an HD video for an hour generates about 3GB of data, so you can imagine how much hundreds of thousands of live and on-demand video classes held daily by these digital fitness companies are producing.

According to the report titled “Global Interactive Fitness Market 2020-2024” by Technavio, the interactive fitness market is expected to grow by USD 5.44 billion during 2020-2024. According to Technavio, the rising trend of “exergaming” is anticipated to boost the growth of the global interactive fitness market.¹ There is already an increased need for memory as we are seeing significant data being generated by these companies who are requiring data quicker and more efficiently.

Across the globe

Source: Mordor Intelligence

Image Download

We expect to see an increased relevance between the memory semiconductor and health businesses with companies continuing to highlight key technologies in big data, AI (Artificial Intelligence) and data centers. According to the report titled “Semiconductor In Healthcare Market – Growth, Trends, And Forecast (2020 – 2025)” by Mordor Intelligence, the semiconductor in healthcare market is expected to grow at a CAGR of over 10.2%, during the forecast period (2020 – 2025), given the increasing investments in research and innovation centers, government programs, and governmental policies that are favoring the IT health equipment and devices markets.² As we can see from cases like Peloton and Mirror, the expansion of the markets will inevitably lead to the explosion of data and growing demand for advanced memory solutions.

Other forms of fitness technology such as fitness trackers and monitoring devices are also contributing to massive data generation boom, where memory chips are also used. These devices are becoming significantly more popular as they can track everything from your heart rate and steps taken to glucose levels for those with diabetes. You could imagine the multiple benefits of these devices offer as they track and log significant data on a daily and even hourly pace to give users valuable personal health information and advice. They also provide users with daily motivations and achievable goals to hold users accountable to live a healthier life.

With the number of these types of devices increasing and functions expanding with each new generation, there will continue to be a strong growth of memory semiconductors in the market across the globe. These memory solutions not only contribute greatly to the devices themselves but in data centers around the world in order to effectively store and properly utilize the information generated.

SK hynix’s 16Gb (Gigabits) DDR5 (Double Data Rate 5) DRAM

Image Download

Memory as a Key

All of this data generated is increasing at a daily rate. As more companies look to join this new growth area, the industry is requiring stable memory solutions with high speed and capacity to properly deliver these offerings to customers who are excited about the possibilities of how technology can enhance their health and fitness experience at home. To do this, analyzing big data is the core of digital fitness and health businesses, and advanced memory solutions in both devices and data centers are key to this vision.

“The live video streaming of digital fitness services realized via AR requires both high-spec user devices and data traffic processing capacity at back-end. It means that to analyze big data utilizing AI, the entire system must be able to process high capacity data,” said Young Choul Kim, Project Leader (PL) at Computing Marketing Team of SK hynix. “At the current stage, the end user devices such as mobile connected devices conduct basic calculations with LPDDR4 DRAM, and data centers receive the results to process more complex analysis with high-performance DDR4 DRAM. In the future, end user devices will adopt High Bandwidth Memory 3 (HBM3) or LPDDR5 DRAM, and DDR5 DRAM will play key roles in back-end processing and analysis in data centers.”

In fact, with diverse factors driving, demands for memory semiconductors from data center clients are surging which helped reduce suppliers’ stock levels in 4Q of 2019. According to the DRAMeXchange research division of TrendForce, 4Q19 NAND flash bit shipment increased by nearly 10% QoQ thanks to demand growth from data center clients.³

As AI and big data greatly expand into more commonplace applications, semiconductors are especially important to reliably store and manage vast amounts of data. Memory devices are used to collect and analyze large amounts of information in real-time.

Kim commented, “As digital revolution expands into various industries, the amount of data needs to be processed is increasing dramatically. It becomes more and more important to utilize the big data to understand the present and foresee the future. As a result, demands from data centers are growing simultaneously.”

Not just a trend

From 2020 onward, it is expected that the memory semiconductor industry will continue to grow with the continued increase in demand from data center clients, supported by various industries including digital fitness and health industry. New startups, such as Tempo, are joining the space to incorporate AI and big data with digital fitness services, while more established companies like Fitbit are expanding their offerings to meet the unique demands from customers.

This is the reason we see a large makeup of the world’s most innovative companies on lists from news outlets like Fast Company and Forbes feature largely HealthTech companies. At SK hynix, we are prepared to meet the needs as predictions for the increased demand for this technology grow into the next decade. We are excited at the possibilities we are seeing from numerous newly arising industries as there are always greater demands for faster and more reliable memory solutions.

¹Jesse Maida, “Global Interactive Fitness Market 2020-2024, Growing Awareness Among People about Healthy Lifestyles to Boost Market Growth”, Technavio. (2020)
²“Semiconductor in Healthcare Market – Growth, Trends, And Forecast (2020 – 2025)”, Mordor Intelligence. (2019)
³Ben Yeh, “Driven by Strong Demand from Data Center Clients, 4Q19 NAND Flash Revenue Grows 8.5%, Says TrendForce”, TrendForce. (2020)

The post Never Work Out Alone! – Interactive Digital Fitness at Home first appeared on SK hynix Newsroom.