A glimpse of SK hynix’s booth at the AI Hardware & Edge AI Summit 2024

SK hynix unveiled an enhanced Accelerator-in-Memory based Accelerator (AiMX) card at the AI Hardware & Edge AI Summit 2024 held September 9–12 in San Jose, California. Organized annually by Kisaco Research, the summit brings together representatives from the AI and machine learning ecosystem to share industry breakthroughs and developments. This year’s event focused on exploring cost and energy efficiency across the entire technology stack.

Marking its fourth appearance at the summit, SK hynix highlighted how its AiM¹ products can boost AI performance across data centers and edge devices².

¹Accelerator in Memory (AiM): SK hynix’s PIM semiconductor product name, which includes GDDR6-AiM.
²Edge device: Hardware that controls the flow of data at the boundary between two networks. While they fulfill numerous roles, edge devices essentially serve as the entry or exit point to a network.

Attendees gather to learn more about the upgraded AimX card

Booth Highlights: Meet the Upgraded AiMX

In the AI era, high-performance memory products are vital for the smooth operation of LLMs³. However, as these LLMs are trained on increasingly larger datasets and continue to expand, there is a growing need for more efficient solutions. SK hynix addresses this demand with its PIM⁴ product AiMX, an AI accelerator card that combines multiple GDDR6-AiMs to provide high bandwidth and outstanding energy efficiency.

³Large language model (LLM): Advanced AI systems that require extensive datasets to train models to understand and generate human-like language. It enables applications like natural language processing and translation.
⁴Processing-In-Memory (PIM): A next-generation technology that embeds processing capabilities within memory, minimizing data transfer between the processor and memory. This boosts efficiency and speed, especially for data-intensive tasks like LLMs, where quick data access and processing are essential.

The 32 GB AiMX prototype card was shown publicly for the first time at the event

At the AI Hardware & Edge AI Summit 2024, SK hynix presented its updated 32 GB AiMX prototype which offers double the capacity of the original card featured at last year’s event. To highlight the new AiMX’s advanced processing capabilities in a multi-batch⁵ environment, SK hynix held a demonstration of the prototype card with the Llama 3⁶ 70B model, an open source LLM. In particular, the demonstration underlined AiMX’s ability to serve as a highly effective attention⁷ accelerator in data centers.

⁵Multi-batch: A computer processing method in which the system groups together multiple tasks (batches) and processes them at once.
⁶Llama 3: An open source LLM developed by Meta, featuring pretrained and instruction-fine-tuned language models.
⁷Attention: Mechanisms which give LLMs context about text, lessening the model’s chance of misunderstandings and allowing it to generate more accurate and contextually relevant outputs.

The upgraded AiMX was demonstrated with the Llama 3 70B model LLM to highlight its processing capabilities

AiMX addresses the cost, performance, and power consumption challenges associated with LLMs in not only data centers, but also in edge devices and on-device AI applications. For example, when applied to mobile on-device AI applications, AiMX improves LLM speed three-fold compared to mobile DRAM while maintaining the same power consumption.

Featured Presentation: Accelerating LLM Services from Data Centers to Edge Devices​

Euicheol Lim presenting on how the AiMX system accelerates LLM services

On the final day of the summit, SK hynix gave a presentation detailing how AiMX is an optimal solution for accelerating LLM services in data centers and edge devices. Euicheol Lim, research fellow and head of the Solution Advanced Technology team, shared the company’s plans to develop AiM products for on-device AI based on mobile DRAM and revealed the future vision for AiM. In closing, Lim emphasized the importance of close collaboration with companies involved in developing and managing data centers and edge systems to further advance AiMX products.

Looking Ahead: SK hynix’s Vision for AiMX in the AI Era

The AI Hardware & Edge AI Summit 2024 provided a platform for SK hynix to demonstrate AiMX’s applications in LLMs across data centers and edge devices. As a low-power, high-speed memory solution able to handle large amounts of data, AiMX is set to play a key role in the advancement of LLMs and AI applications.

The post SK hynix Presents Upgraded AiMX Solution at AI Hardware & Edge AI Summit 2024 first appeared on SK hynix Newsroom.

¹Accelerator-in-Memory based Accelerator (AiMX): SK hynix’s accelerator card product that specializes in large language models (AI that learns with large amounts of text data such as ChatGPT) using GDDR6-AiM chips.

²Accelerator: A special-purpose hardware device that uses a chip designed specifically for processing and computing information.

Figure 1. SK hynix’s exhibition booth at the AI Hardware & Edge AI Summit 2023

Hosted annually by the UK marketing firm Kisaco Research, the AI Hardware & Edge AI Summit brings together global IT companies and high-profile startups to share their developments in artificial intelligence and machine learning. This is SK hynix’s third time participating in the summit.

At the event, the company showcased the prototype of AiMX, an accelerator card that combines multiple GDDR6-AiMs to further enhance performance, along with the GDDR6-AIM itself under the slogan of “Boost Your AI: Discover the Power of PIM³ with SK hynix’s AiM⁴.”

³Processing-In-Memory (PIM): A next-generation technology that adds computational capabilities to semiconductor memories to solve the problem of data movement congestion in AI and big data processing.

⁴Accelerator in Memory (AiM): SK hynix’s PIM semiconductor product name, which includes GDDR6-AiM.

Figure 2. The prototype AiMX card utilizes multiple GDDR6-AiM chips for enhanced performance

As a low-power, high-speed memory solution capable of handling large amounts of data, AiMX is set to play a key role in the advancement of data-intensive generative AI⁵ systems. The performance of generative AI improves as it is trained on more data, highlighting the need for high-performance products which can be applied to an array of generative AI systems.

⁵Generative AI: AI that learns from large amounts of data to actively generate results based on a user’s specific needs.

Figure 3. Demonstrating a large AI language model with AiMX that utilizes GDDR6-AiM

SK hynix also demonstrated Meta’s generative AI Open Pretrained Transformer (OPT) 13B model on a server system equipped with the AiMX prototype. The AiMX system featuring GDDR6-AiM chips reduces data processing time by more than 10 times compared to systems with GPUs, while consuming one-fifth the power. The company’s demonstration piqued the interest of global companies providing AI services by showing that it can deliver higher performance⁶ than the most recent accelerators.

⁶Performance is based on the condition that the AiM Control Hub inside the AiMX card is developed as an application-specific integrated circuit (ASIC).

Figure 4. Eui-cheol Lim, vice president of SK hynix’s Solution Development division, delivers a presentation on AiMX

In addition, the company held a session outlining the benefits of AiMX. In a presentation titled “Cost-Effective Generative AI Inference Acceleration using AiM,” Eui-cheol Lim, vice president of the Solution Development division, compared the performance of GPUs and AiMX and discussed the future of next-generation intelligent semiconductor memories.

“SK hynix’s AiMX is a solution that delivers higher performance while consuming less power, and costing less than conventional GPUs,” Lim explained. “We will continue to develop memory technologies that will lead the way in the era of artificial intelligence.”

The post SK hynix Debuts Prototype of First GDDR6-AiM Accelerator Card ‘AiMX’ for Generative AI first appeared on SK hynix Newsroom.