SK hynix is showcasing its leading AI and data center memory products at the 2024 Open Compute Project (OCP) Global Summit held October 15–17 in San Jose, California. The annual summit brings together industry leaders to discuss advancements in open source hardware and data center technologies. This year, the event’s theme is “From Ideas to Impact,” which aims to foster the realization of theoretical concepts into real-world technologies.

SK hynix’s booth at the 2024 OCP Global Summit

In addition to presenting its advanced memory products at the summit, SK hynix is also strengthening key industry partnerships and sharing its AI memory expertise through insightful presentations. This year, the company is holding eight sessions—up from five in 2023—on topics including HBM¹ and CMS².

¹High Bandwidth Memory (HBM): A high-value, high-performance product that revolutionizes data processing speeds by connecting multiple DRAM chips with through-silicon via (TSV).
²Compute Memory Solution (CMS): A memory solution optimized for AI, HPC, and data centers, enhancing speed, efficiency, and scalability for compute-heavy workloads.

Displays & Demonstrations at the Booth: Transforming AI & Data Centers

Visitors to SK hynix’s booth can see a range of the company’s cutting-edge AI and data center solutions and view demonstrations with high-performance customer systems.

SK hynix’s groundbreaking AI and data center products on display

Among the products being demonstrated is CMM-Ax³, formerly known as CMS 2.0, which was shown under its new name for the first time. For the demonstration, SK hynix is highlighting the product’s role in next-generation compute memory for AI infrastructure, particularly for multi-modal applications. Meanwhile, a heterogeneous memory management solution involving CMM (CXL Memory Module)-DDR5⁴ and HMSDK⁵ is also being demonstrated as well as Niagara 2.0⁶, which is illustrating its efficient utilization of pooled memory resources. Another demonstration is highlighting the computational storage drive’s (CSD) AI storage capabilities for large-scale training systems.

³CXL Memory Module-Ax (CMM-Ax): A high-performance memory module optimized for computational workloads, improving AI and data center efficiency.
⁴CXL Memory Module-DDR5 (CMM-DDR5): A next-gen DDR5 memory module based on CXL that enhances system bandwidth, speed, and performance for AI, cloud, and high-performance computing.
⁵Heterogeneous Memory Software Development Kit (HMSDK): A software development kit specially designed to support CXL memory, a next-generation memory system based on the CXL open industry standard.
⁶Niagara 2.0: An integrated HW/SW solution for pooled memory that allows multiple hosts (CPUs and GPUs) to efficiently share large memory pools to minimize unused or underutilized memory known as stranded memory. By supporting optimal data placement through hot and cold data detection, it can significantly improve system performance.

SK hynix is showcasing CXL-based solutions including CMM-Ax and Niagara 2.0

In addition, a live demonstration of the GDDR6-AiM⁷-based accelerator card AiMX is being held using Meta’s latest large language model (LLM), Llama3 70B, which has 70 billion parameters. The demonstration shows AiMX’s capability to tackle industry challenges. For example, data centers’ LLM services improve GPU efficiency by simultaneously processing requests from multiple users. However, as the length of the generated token increases, the computation of the attention layer⁸ increases, lowering GPU efficiency. Through the demonstration, AiMX shows it can overcome this issue by handling large amounts of data while offering greater efficiency and lower power consumption compared to the latest accelerators.

⁷Accelerator-in-Memory (AiM): SK hynix’s PIM semiconductor product. PIM is next-generation technology that adds computational functions to memory semiconductors, solving the data transfer bottleneck in AI and big data processing fields.
⁸Attention layer: A mechanism which allows a model to determine the importance of input data to focus on more relevant information.

The AiMX card is being demonstrated with Meta’s latest LLM, Llama3 70B

The company also displayed a range of its industry-leading AI memory and data center products. HBM3E is shown along with the NVIDIA H200 Tensor Core GPU and NVIDIA GB200 Grace Blackwell Superchip. The booth also features SK hynix’s DDR5 RDIMM and MCR DIMM server DRAM, as well as enterprise SSDs (eSSDs), with Supermicro servers. The DDR5 products on display include the world’s first DDR5 DRAM built using the 1c node, the sixth generation of the 10nm process technology. Designed to meet the growing computational and energy demands of AI-driven data centers, the 16 Gb 1c DDR5 offers improved performance and energy efficiency from the previous generation. Additionally, SK hynix unveiled the 1bnm 96 GB DDR5 which can reach speeds of up to 7,200 megabits per second (Mbps), making it ideal for supporting massive data flows in data centers.

HBM3E is presented alongside NVIDIA’s H200 and GB200, while DDR5 RDIMM and MCR DIMM products are shown with Supermicro servers

In the SSD section, the Gen5 eSSDs PS1010 and PS1030 and the Gen4 PE9010 are among those on display. Offering ultra-fast read/write speeds, these SSD solutions are vital for accelerating AI training and inference in large-scale environments. Through all these innovations, SK hynix is continuing to lead the way in AI memory and storage solutions, fueling the future of AI and transforming data center operations.

Gen5 eSSDs showcased at the summit

Expanded Presentation Sessions: Sharing AI Memory Expertise

Reflecting SK hynix’s growing influence as the leading AI memory provider, the company is holding eight insightful sessions—three more than in 2023—on the future of AI memory solutions.

SK hynix is sharing its industry expertise through eight presentations

• Youngpyo Joo, head of Software Solution: Joo discussed how AI-driven technological advancements are reshaping computing architecture, focusing on innovations such as CXL^®9-based memory and computational memory solutions for AI workloads.
• Technical Leader Jungmin Choi, Composable System: In his talk, Choi explored how memory disaggregation can improve system efficiency.
• Technical Leader Honggyu Kim, System SW: Kim explained how SK hynix’s HMSDK solution optimizes memory pooling and sharing in AI workloads, improving performance and reducing network congestion.
• Technical Leader Younsoo Kim, DRAM Technology Planning: Kim addressed the future of HBM memory and its pivotal role in AI applications in his presentation.
• Euicheol Lim, head of Solution Advanced Technology (Solution AT): Lim discussed the role of SK hynix’s AiMX card to reduce operational costs in LLM processing in data centers and on-device services.
• Technical Leader Kevin Tang and Team Leader Jongryool Kim, AI System Infra: Tang and Kim delivered a joint presentation about improving LLM training efficiency and scalability through checkpoint¹⁰
• Hoshik Kim, head of SOLAB: As part of a panel discussion, Kim discussed various issues, solutions, and visions that near-data computing¹¹ needs to address for application to various real systems.
• Technical Leader Myoungseo Kim, AI Open Innovation: Kim will deliver the final talk with Vikrant Soman, Solution Architect at Uber, and Jackrabbit Labs CTO Grant Mackey on CMS composable memory architecture and the need for scalable, interoperable memory solutions. The presentation will reveal the results of SK hynix’s CXL pooled memory prototype integration with Jackrabbit Labs’ open source cluster orchestration software to address the pain point of stranded memory in the Uber Data Center Kubernetes Cluster environment.

⁹Compute Express Link (CXL): A PCIe-based next-generation interconnect protocol on which high-performance computing systems are based.
¹⁰Checkpoint: A technology that stores model parameters and related key data at a specific point during the learning (training) process, enabling the learning process to restart from the saved point in case of a system failure.
¹¹Near-data computing: A computing method aimed at addressing the bottleneck issue between the memory and processor, a limitation of the von Neumann architecture. It delivers only refined data processed by memory to the processor to minimize the movement of data within the system, improving performance and cost efficiency.

Shaping the Future: Advancing AI Memory & Data Center Innovation

At the 2024 OCP Global Summit, SK hynix is reinforcing its industry leadership by not only showcasing its advanced AI memory and data center solutions but also sharing its expertise. Looking ahead, the company is dedicated to pioneering breakthroughs that will define the next generation of AI and data center technologies.

The post SK hynix at the 2024 OCP Global Summit: Leading the Future of AI & Data Center Memory Solutions first appeared on SK hynix Newsroom.

SK hynix showcased its next-generation memory semiconductor technologies and solutions at the OCP Global Summit 2023 held in San Jose, California from October 17–19.

The OCP Global Summit is an annual event hosted by the world’s largest data center technology community, the Open Compute Project (OCP), where industry experts gather to share various technologies and visions. This year, SK hynix and its subsidiary Solidigm showcased advanced semiconductor memory products that will lead the AI era under the slogan “United Through Technology”.

▲ Figure 1. SK hynix’s exhibition booth at the OCP Global Summit 2023

At the Booth: Leading Global AI Memory Technologies in the Spotlight

▲ Figure 2. HBM(HBM3/HBM3E), MCR DIMM, DDR5 RDIMM, and LPDDR CAMM products on display at SK hynix’s booth

SK hynix presented a broad range of its solutions at the summit, including its leading HBM¹(HBM3/3E), CXL², and AiM³ products for generative AI. The company also unveiled some of the latest additions to its product portfolio including its DDR5 RDIMM, MCR DIMM, enterprise SSD (eSSD), and LPDDR CAMM⁴ devices.

¹High Bandwidth Memory (HBM): A high-value, high-performance product that possesses much higher data processing speeds compared to existing DRAMs by vertically connecting multiple DRAMs with through-silicon via (TSV).
²Compute Express Link (CXL): A next-generation memory solution that increases the memory bandwidth of server systems with traditional DRAM products to improve performance and easily expand memory capacity.
³Accelerator in Memory (AiM): SK hynix’s PIM semiconductor product name, which includes GDDR6-AiM.
⁴Low Power Double Data Rate Compression Attached Memory Module (LPDDR CAMM): A solution developed based on the LPDDR package in line with the next-generation memory standard (CAMM) for laptops and mobile devices. Compared to conventional So-DIMM modules, LPDDR CAMM has a single-sided configuration which is half as thick and offers improved capacity and power efficiency.

Visitors to the HBM exhibit could see HBM3, which is utilized in NVIDIA’s H100, a high-performance GPU for AI, and also check out the next-generation HBM3E. Due to their low-power consumption and ultra-high-performance, these HBM solutions are more eco-friendly and are particularly suitable for power-hungry AI server systems.

▲ Figure 3. CXL-based CMS 2.0, pooled memory, and memory expander solutions being demonstrated at the event

Three of SK hynix’s key CXL products were demonstrated at the event, including its CXL-based computational memory solution (CMS)⁵ 2.0. A next-generation memory solution that integrates computational functions into CXL memory, CMS 2.0 uses NMP⁶ to minimize data movement between the CPU and memory. CMS 2.0 was applied to SK Telecom’s spatial data analysis and visualization solution based on Lightning DB which analyzes foot traffic in real time, illustrating CMS 2.0’s data processing power equivalent to a server CPU. Moreover, the demonstration highlighted that CMS 2.0 can improve data processing performance and energy efficiency.

⁵Computational Memory Solution (CMS):  A memory solution that offers the functions of machine learning and data filtering, which are frequently performed by big data analytics applications. Just like CXL, CMS’s memory capacity is highly scalable.
⁶Near-Memory Processing (NMP): A memory architecture which moves the compute capability next to the main memory to address CPU memory bottlenecks and improve processing performance.

The company also demonstrated its CXL-based pooled memory solution which can significantly reduce idle memory usage and shorten the overhead time of data movement in distributed processing environments such as AI and big data. The demonstration, which featured a technological collaboration with software provider MemVerge, showcased how CXL-based pooled memory solutions can improve system performance in AI and big data distributed processing systems.

Additionally, SK hynix demonstrated its CXL-based memory expander applied to Meta’s software caching engine, CacheLib, which showed how the CXL-based memory solution optimizes software for improved performance and cost savings.

▲ Figure 4. PIM and AiMX on display at the summit

AiM, a game-changing technology that redefines memory’s role in AI inference, was also presented at the booth. A subset of SK hynix’s PIM⁷ portfolio, AiM promises to revolutionize machine learning, high-performance computing, and big data applications while reducing operating costs. The company demonstrated its GDDR6-AiM, a solution which incorporates computational capabilities into the memory and improves the performance and efficiency of data-intensive generative AI inference systems. Additionally, a demonstration was held for the prototype of AiMX⁸, an innovative generative AI accelerator card based on GDDR6-AiM.

⁷Processing-In-Memory (PIM): An advanced technology that combines computational capabilities and memory within just one die to perform the best computing capability.
⁸Accelerator-in-Memory based Accelerator (AiMX): SK hynix’s accelerator card featuring a GDDR6-AiM chip which is specialized for large language models.

▲ Figure 5. PS1010 E3.S, a PCIe Gen5 eSSD product, showcased at the booth

SK hynix also showcased the PS1010 E3.S, a PCIe (Peripheral Component Interconnect Express) Gen5-based eSSD. Designed for data-intensive applications, cloud computing, and AI workloads, the PS1010 E3.S promises improved performance and reliability with superior speed and lower carbon emissions compared to previous generations.

At the Sessions: Sharing Industry Insights & Key Technologies

▲ Figure 6. SK hynix’s Hoshik Kim, fellow of the System Architecture Group in Memory Forest x&D, holds a session on CXL technology

Through thought-provoking talks and sessions at the summit, SK hynix also suggested the future development of next-generation memory solutions.

In a panel session titled “Data Central Computing, Present and Future”, Hoshik Kim, fellow of the System Architecture Group in Memory Forest x&D, joined other industry experts to omputing and programming models. Kim also held an executive session in which he revealed SK hynix’s progress in developing CXL technology and shared the company’s vision under the theme of “CXL: A Prelude to a Memory-Centric Computing”.

▲ Figure 7. SK hynix’s Youngpyo Joo, Dongwuk Moon and Jungmin Choi of Memory Forest x&D giving talks on CXL-based solutions

The potential of CXL was also the focal point of three other sessions held by the company’s employees from the Memory x&D department. Youngpyo Joo, head of the Software Solutions Group, explored how CXL-based computational memory solution architecture can offer increased usability and flexibility.

Meanwhile, Donguk Moon, technical leader of the Platform Software team, introduced how emerging CXL devices can enhance memory capacity and bandwidth at the caching layer for technologies such as AI and web services. In addition, Jungmin Choi, technical leader of the Composable Memory team, covered how the latest CXL technology will further support the disaggregation of memory. Choi also emphasized that this technology can not only solve problems currently faced by data centers such as idle memory, but also improve system performance.

▲ Figure 8. Euicheol Lim, head of SK hynix’s Memory Solution Product Design Group, discusses PIM-based AiM and PIM technologies

In another engaging session, Euicheol Lim, head of the Memory Solution Product Design Group, explored the potential for PIM technology to meet the high memory and computational demands of AI. Lim highlighted key technologies based on PIM, including SK hynix’s PIM-based AiM accelerator.

Advanced Solutions to Unlock Next-Generation Technologies

As the OCP Global Summit 2023 drew to a close, SK hynix reinforced its commitment to developing groundbreaking solutions which can help realize advanced technologies such as AI. Going forward, the company will continue to tackle industry challenges and make technological breakthroughs for the AI era as the global no.1 AI memory solution provider.

The post SK hynix Displays Next-Gen Solutions Set to Unlock AI & More at OCP Global Summit 2023 first appeared on SK hynix Newsroom.

The Open Compute Project (OCP) is an initiative that proposes a new direction in which both software and hardware disciplines are tightly connected in order to efficiently address the increasing requirements of compute infrastructure. It gathers major decision-makers, technologists, engineer developers, and suppliers to exchange insights, discuss challenges, and provide solutions in and around the data center.

The OCP is also characterized by open source and open collaboration, with a global committee covering everywhere from the telecommunications industry to edge infrastructure, and everything in between.

The OCP Global Summit, held in the Fall each year, provides a unique opportunity to see and explore the various ways in which the data center industry is innovating and evolving. SK hynix attended the 2022 OCP Global Summit in San Jose, California, from October 18 to 20, to demonstrate its latest products, technologies, and ideas.

SK hynix, a Proud Supporter of OCP

Image 1. SK hynix’s booth at OCP Global Summit 2022 drew enthusiasm from participants and visitors

As a Ruby Sponsor, SK hynix set up a booth at OCP with the theme “Driving New Level of Performance for your Ultimate Data Experience,” showcasing latest research developments in next-generation memory technologies. Having attended every year since 2016, this year’s event was particularly memorable as it reverted to a face-to-face format for the first time since the Covid pandemic began.

At the event, the company unveiled the PCIe Gen5 PS1010 for the first time as well as highlighted offerings such as the DDR5 DRAM-based CXL (Compute Express Link) memory solution and next-generation intelligent semiconductor GDDR6-AiM, demonstrating how the company has truly become a total solution provider.

Demonstrating a Complete Solution Portfolio

Image 2. SK hynix demonstrated a portfolio of products on-site at the OCP Global Summit 2022 including a first look at PS1010

Four products were demonstrated by SK hynix at the OCP Global Summit 2022, starting with the PS1010. The PS1010 is a PCle (Peripheral Component Interconnect Express) Gen5 total solution product that handles all SoC/DRAM/NAND and operations in-house, with data transfer speeds more than doubling over the previous generation due to the usage of the PCIe Gen5 interface. It was the first time SK hynix unveiled the product, emphasizing its use of industry-leading V7 NAND to ensure cost and performance competitiveness.

Second to be featured was the company’s DDR5 DRAM-based CXL memory solution that integrates computational functions into CXL memory for the first time in the industry. The solution is expected to be installed on next-generation server platforms in order to improve system performance and energy efficiency.

Third, a sample of the EDSFF (Enterprise & Data Center Standard Form Factor) E3.S was demonstrated following the announcement of the development of the latest DDR5 24Gb DRAM-based 96GB CXL memory sample. The combination of DDR5 and CXL memory was shown to reduce memory execution time by 20% and increase bandwidth by 50%, resulting in a 60% memory capacity expansion impact.

Finally, a sample of GDDR6-AiM (Accelerator in Memory) demonstrated the behavior of a huge AI model aimed at memory-centric computing. GDDR6-AiM combines processing function onto a typical DRAM, and can reduce energy consumption by up to 80% while providing up to 16 times the performance and lower operating voltage.

Thought Leadership in Memory Solutions

Image 3. Thomas Won Ha Choi sharing SK hynix’s perspective on the value of its new CXL memory

Besides the demonstrations that attracted a great deal of attention, SK hynix also participated in three session discussions at the OCP Global Summit. Thomas Won Ha Choi, Director of DRAM Product Planning at SK hynix, shared the company’s perspective on the new value of CXL memory, as well as its vision for planning and enabling the future development of robust CXL memory. He presented SK hynix’s various memory solutions to provide examples of how CXL devices can enhance memory system values.

Image 4. Donguk Moon introduces SK hynix’s CXL-based CMS at his session discussion

Donguk Moon, SK hynix’s Senior Technical Program Manager for Memory System Research, also introduced SK hynix’s CXL-based CMS, as well as collaboration with SK Telecom, one of Korea’s major telecommunications companies, to improve their Lightning DB real-time data management system by incorporating CMS into it.

A Future-Oriented Approach to Data Center Research

While SK hynix has concluded its successful participation in the OCP Global Summit this year, the company’s research continues, beyond the contents presented in the last few days. Next year, SK hynix looks forward to continuing to present important development achievements and new products for the future.

Observing the enthusiasm and passion shown by participants and visitors was inspiring, and further motivates SK hynix’s determination to serve as the ultimate total solution provider for the data center industry.

The post SK hynix Successfully Establishes Itself as a Total Solution Provider at OCP Global Summit 2022 first appeared on SK hynix Newsroom.

Just two months after introducing the first CXL memory sample in August, SK hynix has successfully developed a computational memory solution, becoming the industry’s first to integrate computational functions into CXL memory. The solution, a result of a close collaboration with SK Telecom from early on for optimization of the next-generation, high-performance computing infra system, marks a meaningful case where SK Group’s ICT companies converged their technologies for a joint development.

Unveiling both the CMS and its software platform at the Open Compute Project (OCP) Global Summit 2022 taking place from Oct. 18 in San Jose, USA, SK hynix showed a next-generation high-performance computing infrastructure solution as well as the value of the solution from the customer’s perspective. The solution is expected to be installed on next-generation server platforms to improve system performance as well as energy efficiency.

Integrated Solution for Memory, Computational Functions Introduced

CXL (Compute eXpress Link)*, a base technology for the latest development, is a new interconnect technology designed to promote efficiency of various solutions such as GPU, AI accelerator and  memory.

*CXL(Compute Express Link): A PCIe-based next-generation interconnect protocol on which high-performance computing systems are based

Among the advantages of CXL that have caught the attention of the industry is its flexible increase in memory capacity. CXL enables the flexible expansion of a system’s memory capacity by installing a memory card like a GPU or SSD. SK hynix focused on CXL’s technological characteristic of supporting both memory and accelerator to proceed with a preemptive research and development and, consequently, come up with the latest integrated solution. Park Kyoung, Head of Memory System Research at SK hynix, noted, “CXL has created an opportunity for memory providers.”

The CMS technology provides not only the advantage of CXL but also showcases advantages of machine learning and data filtering computation functions frequently performed by big data analysis applications. The solution also screens operations that are inefficient for the CPU to perform or that consume a lot of energy during the transfer of data between the memory and the CPU. Park Kyoung, announcing follow-up research and development plans, said, “Through the internalization of computational functions, CMS enabled performance several times faster than that of dozens of CPU cores in specific computations. Considering that this is just a prototype, we think we can improve the performance even further and are considering applying the technology to other applications such as big data.”

Convergence of SK Group companies’ leading technologies

Park also said, “When introducing new technologies and their solutions to the market, it is important to prove their value from the customer’s point of view, but the possible scope of collaboration with customers is relatively limited, especially for data-center server areas. In that sense, collaboration with SK Telecom provided a great opportunity to bring the CMS into reality. SK Telecom’s experience in AI/Big Data-based service infrastructure development and operations was reflected in each part of the development process.” In order to efficiently process large amounts of data, SK Telecom has developed and ran in various commercial services its own Lightning DB*, an in-memory data analytics platform.

*Lightning DB: SK Telecom’s platform with a memory-centric design specialized in real-time data processing. It can carry on high data analysis performance through optimized parallel processing after dividing and storing large amounts of data in smaller units.

The two companies proceeded with development of both CMS and CMS-applied Lightning DB simultaneously. Through this process, SK hynix was able to discover and apply the computational functions required by customers, and SK Telecom was able to enhance the competitiveness of its platform.

Yang Seung-ji, Vice President and Head of Vision R&D at SK Telecom, said, “It is common for hardware solutions of a new concept to take a considerable amount of time to prove their effectiveness from a software application’s point of view. But this time, meaningful computations for SK Telecom’s actual application services were selected, which helped us save a significant amount of time by jointly performing all processes from structure design of hardware-software to development and verification.” Yang continued, “As we verified the performance improvements of the solutions, we plan to apply them to screening tasks that increase accuracy in large-scale AI learning data in the future and use them to strengthen the competitiveness of SK Telecom’s AI service.”

SK Group’s ICT affiliates have been operating R&D projects for synergies to converge various technologies of related companies that encompass semiconductors, hardware solutions, software platforms, and AI services. Lee Jong-min, Vice President and Head of Future R&D at SK Telecom, said, “This case is significant in that we successfully developed a leading technology by combining AI and semiconductor capabilities of SK Telecom and SK hynix together. We will continue to provide new technology-based value to customers and contribute to the ICT R&D ecosystem globally.”

The CXL memory solution prototype developed by SK hynix and the big data analysis platform optimized together with SK Telecom will be unveiled initially at the Open Compute Project (OCP) Global Summit and will also be demonstrated at the SK Tech Summit in Korea in early November. In the future, SK hynix plans to actively conduct R&D and collaborations related to CXL memory that provides added value from the customer’s point of view.

The post SK hynix Introduces Industry’s First CXL-based Computational Memory Solution (CMS) at the OCP Global Summit first appeared on SK hynix Newsroom.