SK hynix is showcasing its leading AI and data center memory products at the 2024 Open Compute Project (OCP) Global Summit held October 15–17 in San Jose, California. The annual summit brings together industry leaders to discuss advancements in open source hardware and data center technologies. This year, the event’s theme is “From Ideas to Impact,” which aims to foster the realization of theoretical concepts into real-world technologies.

SK hynix’s booth at the 2024 OCP Global Summit

In addition to presenting its advanced memory products at the summit, SK hynix is also strengthening key industry partnerships and sharing its AI memory expertise through insightful presentations. This year, the company is holding eight sessions—up from five in 2023—on topics including HBM¹ and CMS².

¹High Bandwidth Memory (HBM): A high-value, high-performance product that revolutionizes data processing speeds by connecting multiple DRAM chips with through-silicon via (TSV).
²Compute Memory Solution (CMS): A memory solution optimized for AI, HPC, and data centers, enhancing speed, efficiency, and scalability for compute-heavy workloads.

Displays & Demonstrations at the Booth: Transforming AI & Data Centers

Visitors to SK hynix’s booth can see a range of the company’s cutting-edge AI and data center solutions and view demonstrations with high-performance customer systems.

SK hynix’s groundbreaking AI and data center products on display

Among the products being demonstrated is CMM-Ax³, formerly known as CMS 2.0, which was shown under its new name for the first time. For the demonstration, SK hynix is highlighting the product’s role in next-generation compute memory for AI infrastructure, particularly for multi-modal applications. Meanwhile, a heterogeneous memory management solution involving CMM (CXL Memory Module)-DDR5⁴ and HMSDK⁵ is also being demonstrated as well as Niagara 2.0⁶, which is illustrating its efficient utilization of pooled memory resources. Another demonstration is highlighting the computational storage drive’s (CSD) AI storage capabilities for large-scale training systems.

³CXL Memory Module-Ax (CMM-Ax): A high-performance memory module optimized for computational workloads, improving AI and data center efficiency.
⁴CXL Memory Module-DDR5 (CMM-DDR5): A next-gen DDR5 memory module based on CXL that enhances system bandwidth, speed, and performance for AI, cloud, and high-performance computing.
⁵Heterogeneous Memory Software Development Kit (HMSDK): A software development kit specially designed to support CXL memory, a next-generation memory system based on the CXL open industry standard.
⁶Niagara 2.0: An integrated HW/SW solution for pooled memory that allows multiple hosts (CPUs and GPUs) to efficiently share large memory pools to minimize unused or underutilized memory known as stranded memory. By supporting optimal data placement through hot and cold data detection, it can significantly improve system performance.

SK hynix is showcasing CXL-based solutions including CMM-Ax and Niagara 2.0

In addition, a live demonstration of the GDDR6-AiM⁷-based accelerator card AiMX is being held using Meta’s latest large language model (LLM), Llama3 70B, which has 70 billion parameters. The demonstration shows AiMX’s capability to tackle industry challenges. For example, data centers’ LLM services improve GPU efficiency by simultaneously processing requests from multiple users. However, as the length of the generated token increases, the computation of the attention layer⁸ increases, lowering GPU efficiency. Through the demonstration, AiMX shows it can overcome this issue by handling large amounts of data while offering greater efficiency and lower power consumption compared to the latest accelerators.

⁷Accelerator-in-Memory (AiM): SK hynix’s PIM semiconductor product. PIM is next-generation technology that adds computational functions to memory semiconductors, solving the data transfer bottleneck in AI and big data processing fields.
⁸Attention layer: A mechanism which allows a model to determine the importance of input data to focus on more relevant information.

The AiMX card is being demonstrated with Meta’s latest LLM, Llama3 70B

The company also displayed a range of its industry-leading AI memory and data center products. HBM3E is shown along with the NVIDIA H200 Tensor Core GPU and NVIDIA GB200 Grace Blackwell Superchip. The booth also features SK hynix’s DDR5 RDIMM and MCR DIMM server DRAM, as well as enterprise SSDs (eSSDs), with Supermicro servers. The DDR5 products on display include the world’s first DDR5 DRAM built using the 1c node, the sixth generation of the 10nm process technology. Designed to meet the growing computational and energy demands of AI-driven data centers, the 16 Gb 1c DDR5 offers improved performance and energy efficiency from the previous generation. Additionally, SK hynix unveiled the 1bnm 96 GB DDR5 which can reach speeds of up to 7,200 megabits per second (Mbps), making it ideal for supporting massive data flows in data centers.

HBM3E is presented alongside NVIDIA’s H200 and GB200, while DDR5 RDIMM and MCR DIMM products are shown with Supermicro servers

In the SSD section, the Gen5 eSSDs PS1010 and PS1030 and the Gen4 PE9010 are among those on display. Offering ultra-fast read/write speeds, these SSD solutions are vital for accelerating AI training and inference in large-scale environments. Through all these innovations, SK hynix is continuing to lead the way in AI memory and storage solutions, fueling the future of AI and transforming data center operations.

Gen5 eSSDs showcased at the summit

Expanded Presentation Sessions: Sharing AI Memory Expertise

Reflecting SK hynix’s growing influence as the leading AI memory provider, the company is holding eight insightful sessions—three more than in 2023—on the future of AI memory solutions.

SK hynix is sharing its industry expertise through eight presentations

• Youngpyo Joo, head of Software Solution: Joo discussed how AI-driven technological advancements are reshaping computing architecture, focusing on innovations such as CXL^®9-based memory and computational memory solutions for AI workloads.
• Technical Leader Jungmin Choi, Composable System: In his talk, Choi explored how memory disaggregation can improve system efficiency.
• Technical Leader Honggyu Kim, System SW: Kim explained how SK hynix’s HMSDK solution optimizes memory pooling and sharing in AI workloads, improving performance and reducing network congestion.
• Technical Leader Younsoo Kim, DRAM Technology Planning: Kim addressed the future of HBM memory and its pivotal role in AI applications in his presentation.
• Euicheol Lim, head of Solution Advanced Technology (Solution AT): Lim discussed the role of SK hynix’s AiMX card to reduce operational costs in LLM processing in data centers and on-device services.
• Technical Leader Kevin Tang and Team Leader Jongryool Kim, AI System Infra: Tang and Kim delivered a joint presentation about improving LLM training efficiency and scalability through checkpoint¹⁰
• Hoshik Kim, head of SOLAB: As part of a panel discussion, Kim discussed various issues, solutions, and visions that near-data computing¹¹ needs to address for application to various real systems.
• Technical Leader Myoungseo Kim, AI Open Innovation: Kim will deliver the final talk with Vikrant Soman, Solution Architect at Uber, and Jackrabbit Labs CTO Grant Mackey on CMS composable memory architecture and the need for scalable, interoperable memory solutions. The presentation will reveal the results of SK hynix’s CXL pooled memory prototype integration with Jackrabbit Labs’ open source cluster orchestration software to address the pain point of stranded memory in the Uber Data Center Kubernetes Cluster environment.

⁹Compute Express Link (CXL): A PCIe-based next-generation interconnect protocol on which high-performance computing systems are based.
¹⁰Checkpoint: A technology that stores model parameters and related key data at a specific point during the learning (training) process, enabling the learning process to restart from the saved point in case of a system failure.
¹¹Near-data computing: A computing method aimed at addressing the bottleneck issue between the memory and processor, a limitation of the von Neumann architecture. It delivers only refined data processed by memory to the processor to minimize the movement of data within the system, improving performance and cost efficiency.

Shaping the Future: Advancing AI Memory & Data Center Innovation

At the 2024 OCP Global Summit, SK hynix is reinforcing its industry leadership by not only showcasing its advanced AI memory and data center solutions but also sharing its expertise. Looking ahead, the company is dedicated to pioneering breakthroughs that will define the next generation of AI and data center technologies.

The post SK hynix at the 2024 OCP Global Summit: Leading the Future of AI & Data Center Memory Solutions first appeared on SK hynix Newsroom.

The Open Compute Project (OCP) is an initiative that proposes a new direction in which both software and hardware disciplines are tightly connected in order to efficiently address the increasing requirements of compute infrastructure. It gathers major decision-makers, technologists, engineer developers, and suppliers to exchange insights, discuss challenges, and provide solutions in and around the data center.

The OCP is also characterized by open source and open collaboration, with a global committee covering everywhere from the telecommunications industry to edge infrastructure, and everything in between.

The OCP Global Summit, held in the Fall each year, provides a unique opportunity to see and explore the various ways in which the data center industry is innovating and evolving. SK hynix attended the 2022 OCP Global Summit in San Jose, California, from October 18 to 20, to demonstrate its latest products, technologies, and ideas.

SK hynix, a Proud Supporter of OCP

Image 1. SK hynix’s booth at OCP Global Summit 2022 drew enthusiasm from participants and visitors

As a Ruby Sponsor, SK hynix set up a booth at OCP with the theme “Driving New Level of Performance for your Ultimate Data Experience,” showcasing latest research developments in next-generation memory technologies. Having attended every year since 2016, this year’s event was particularly memorable as it reverted to a face-to-face format for the first time since the Covid pandemic began.

At the event, the company unveiled the PCIe Gen5 PS1010 for the first time as well as highlighted offerings such as the DDR5 DRAM-based CXL (Compute Express Link) memory solution and next-generation intelligent semiconductor GDDR6-AiM, demonstrating how the company has truly become a total solution provider.

Demonstrating a Complete Solution Portfolio

Image 2. SK hynix demonstrated a portfolio of products on-site at the OCP Global Summit 2022 including a first look at PS1010

Four products were demonstrated by SK hynix at the OCP Global Summit 2022, starting with the PS1010. The PS1010 is a PCle (Peripheral Component Interconnect Express) Gen5 total solution product that handles all SoC/DRAM/NAND and operations in-house, with data transfer speeds more than doubling over the previous generation due to the usage of the PCIe Gen5 interface. It was the first time SK hynix unveiled the product, emphasizing its use of industry-leading V7 NAND to ensure cost and performance competitiveness.

Second to be featured was the company’s DDR5 DRAM-based CXL memory solution that integrates computational functions into CXL memory for the first time in the industry. The solution is expected to be installed on next-generation server platforms in order to improve system performance and energy efficiency.

Third, a sample of the EDSFF (Enterprise & Data Center Standard Form Factor) E3.S was demonstrated following the announcement of the development of the latest DDR5 24Gb DRAM-based 96GB CXL memory sample. The combination of DDR5 and CXL memory was shown to reduce memory execution time by 20% and increase bandwidth by 50%, resulting in a 60% memory capacity expansion impact.

Finally, a sample of GDDR6-AiM (Accelerator in Memory) demonstrated the behavior of a huge AI model aimed at memory-centric computing. GDDR6-AiM combines processing function onto a typical DRAM, and can reduce energy consumption by up to 80% while providing up to 16 times the performance and lower operating voltage.

Thought Leadership in Memory Solutions

Image 3. Thomas Won Ha Choi sharing SK hynix’s perspective on the value of its new CXL memory

Besides the demonstrations that attracted a great deal of attention, SK hynix also participated in three session discussions at the OCP Global Summit. Thomas Won Ha Choi, Director of DRAM Product Planning at SK hynix, shared the company’s perspective on the new value of CXL memory, as well as its vision for planning and enabling the future development of robust CXL memory. He presented SK hynix’s various memory solutions to provide examples of how CXL devices can enhance memory system values.

Image 4. Donguk Moon introduces SK hynix’s CXL-based CMS at his session discussion

Donguk Moon, SK hynix’s Senior Technical Program Manager for Memory System Research, also introduced SK hynix’s CXL-based CMS, as well as collaboration with SK Telecom, one of Korea’s major telecommunications companies, to improve their Lightning DB real-time data management system by incorporating CMS into it.

A Future-Oriented Approach to Data Center Research

While SK hynix has concluded its successful participation in the OCP Global Summit this year, the company’s research continues, beyond the contents presented in the last few days. Next year, SK hynix looks forward to continuing to present important development achievements and new products for the future.

Observing the enthusiasm and passion shown by participants and visitors was inspiring, and further motivates SK hynix’s determination to serve as the ultimate total solution provider for the data center industry.

The post SK hynix Successfully Establishes Itself as a Total Solution Provider at OCP Global Summit 2022 first appeared on SK hynix Newsroom.

Just two months after introducing the first CXL memory sample in August, SK hynix has successfully developed a computational memory solution, becoming the industry’s first to integrate computational functions into CXL memory. The solution, a result of a close collaboration with SK Telecom from early on for optimization of the next-generation, high-performance computing infra system, marks a meaningful case where SK Group’s ICT companies converged their technologies for a joint development.

Unveiling both the CMS and its software platform at the Open Compute Project (OCP) Global Summit 2022 taking place from Oct. 18 in San Jose, USA, SK hynix showed a next-generation high-performance computing infrastructure solution as well as the value of the solution from the customer’s perspective. The solution is expected to be installed on next-generation server platforms to improve system performance as well as energy efficiency.

Integrated Solution for Memory, Computational Functions Introduced

CXL (Compute eXpress Link)*, a base technology for the latest development, is a new interconnect technology designed to promote efficiency of various solutions such as GPU, AI accelerator and  memory.

*CXL(Compute Express Link): A PCIe-based next-generation interconnect protocol on which high-performance computing systems are based

Among the advantages of CXL that have caught the attention of the industry is its flexible increase in memory capacity. CXL enables the flexible expansion of a system’s memory capacity by installing a memory card like a GPU or SSD. SK hynix focused on CXL’s technological characteristic of supporting both memory and accelerator to proceed with a preemptive research and development and, consequently, come up with the latest integrated solution. Park Kyoung, Head of Memory System Research at SK hynix, noted, “CXL has created an opportunity for memory providers.”

The CMS technology provides not only the advantage of CXL but also showcases advantages of machine learning and data filtering computation functions frequently performed by big data analysis applications. The solution also screens operations that are inefficient for the CPU to perform or that consume a lot of energy during the transfer of data between the memory and the CPU. Park Kyoung, announcing follow-up research and development plans, said, “Through the internalization of computational functions, CMS enabled performance several times faster than that of dozens of CPU cores in specific computations. Considering that this is just a prototype, we think we can improve the performance even further and are considering applying the technology to other applications such as big data.”

Convergence of SK Group companies’ leading technologies

Park also said, “When introducing new technologies and their solutions to the market, it is important to prove their value from the customer’s point of view, but the possible scope of collaboration with customers is relatively limited, especially for data-center server areas. In that sense, collaboration with SK Telecom provided a great opportunity to bring the CMS into reality. SK Telecom’s experience in AI/Big Data-based service infrastructure development and operations was reflected in each part of the development process.” In order to efficiently process large amounts of data, SK Telecom has developed and ran in various commercial services its own Lightning DB*, an in-memory data analytics platform.

*Lightning DB: SK Telecom’s platform with a memory-centric design specialized in real-time data processing. It can carry on high data analysis performance through optimized parallel processing after dividing and storing large amounts of data in smaller units.

The two companies proceeded with development of both CMS and CMS-applied Lightning DB simultaneously. Through this process, SK hynix was able to discover and apply the computational functions required by customers, and SK Telecom was able to enhance the competitiveness of its platform.

Yang Seung-ji, Vice President and Head of Vision R&D at SK Telecom, said, “It is common for hardware solutions of a new concept to take a considerable amount of time to prove their effectiveness from a software application’s point of view. But this time, meaningful computations for SK Telecom’s actual application services were selected, which helped us save a significant amount of time by jointly performing all processes from structure design of hardware-software to development and verification.” Yang continued, “As we verified the performance improvements of the solutions, we plan to apply them to screening tasks that increase accuracy in large-scale AI learning data in the future and use them to strengthen the competitiveness of SK Telecom’s AI service.”

SK Group’s ICT affiliates have been operating R&D projects for synergies to converge various technologies of related companies that encompass semiconductors, hardware solutions, software platforms, and AI services. Lee Jong-min, Vice President and Head of Future R&D at SK Telecom, said, “This case is significant in that we successfully developed a leading technology by combining AI and semiconductor capabilities of SK Telecom and SK hynix together. We will continue to provide new technology-based value to customers and contribute to the ICT R&D ecosystem globally.”

The CXL memory solution prototype developed by SK hynix and the big data analysis platform optimized together with SK Telecom will be unveiled initially at the Open Compute Project (OCP) Global Summit and will also be demonstrated at the SK Tech Summit in Korea in early November. In the future, SK hynix plans to actively conduct R&D and collaborations related to CXL memory that provides added value from the customer’s point of view.

The post SK hynix Introduces Industry’s First CXL-based Computational Memory Solution (CMS) at the OCP Global Summit first appeared on SK hynix Newsroom.