In this latest article from our series with the DGIST faculty, Professor Junghyup Lee of the Department of Electrical Engineering and Computer Science will introduce sensor systems that are ubiquitous around us. The article discusses how sensor systems detect data from its environment and converts them into digital signals that computers can store and process. There will also be an emphasis on sensor interface circuits which can be implemented using semiconductor technologies.

The Two Pillars of a Sensor System: Sensor Devices and Sensor Interfaces

Figure 1. The various inputs and outputs of a sensor interface circuit

The sensors that are widely used today are generally referred to as sensor systems that consist of a sensor device and sensor interface. This system detects physical, chemical, and biological data from the environment such as temperature, humidity, motion, pressure, physical contact, and gas to convert into electrical signals.

Various types of sensor devices have been developed with consideration of different principles, structures, and materials that have different targets to be sensed. Temperature sensors, for example, are further classified into different types such as resistance temperature detectors (RTD) and thermocouples based on the measuring methods used, and there are far more types of temperature sensor devices when considering additional factors such as component materials and structures. But even if these sensor devices are divided into many different types, their outputs are ultimately determined by one of four electrical components of voltage, current, resistance, or capacitance. In summary, a sensor device converts a variety of data from the physical world into an analog signal that is in the form of one of these four electrical components.

Afterwards, a sensor interface circuit converts the electrical analog signals from the sensor devices into digital signals for hardware like CPUs, GPUs, monitors, and transceivers to use for signal processing, analysis, and transmission. There are four different types of sensor interface circuits depending on the output of the sensor device: analog-to-digital converters (ADC), current-to-digital converters (CDC), resistance-to-digital converters (RDC), and capacitance-to-digital converters (CDC). As for ADCs, they make digital signals from voltage analog signal.

Even though sensor interface circuits do not require large bandwidths to read signals deriving from physical, chemical, and biological data due to these signals primarily existing in frequencies of tens of kHz or less, it becomes necessary to accurately read very small but meaningful signals amidst the presence of large, external noise. As an example, brainwave signals measure only a few microvolts (μV). However, on the other hand, a 220V power cable nearby can cause a noise signal measuring 60 Hz which equates to several hundred millivolts (mV). In order to accurately detect such small signals like brainwaves in the presence of large distractions like a power cable, sensor interface circuits are very much required to have a wide input range, good linearity, low noise capabilities, and high resolution. Consequently, it is important for sensor systems to also possess the characteristic of low power consumption. In addition to a large number of sensors being found in mobile devices like the latest smartphones, sensor systems need to be readily applied to low-power systems such as Internet of Things (IoT) or Internet of Everything (IoE) products.

Semiconductor-Based Sensor Interfaces: Combining an AFE and ADC

Figure 2. The various inputs and characteristics of a sensor interface

As a system that can be implemented with semiconductors, sensor interfaces typically combine an analog front-end (AFE) circuit and an ADC circuit. The role of the AFE is to convert the sensor’s output signals into voltage signals without loss or distortion and input them to the ADC. The ADC then accurately converts these analog voltage signals into the final digital values. And for sensor interfaces to possess its desired characteristics, it is critical for the implementation of a semiconductor circuit to balance the distribution of performance between the AFE and the ADC to achieve the optimal performance with a high figure-of-merit¹.

¹Figure-of-merit: a numerical quantity based on one or more characteristics of a system or device that represents a measure of efficiency or effectiveness

AFE: Converting Voltage, Current, Resistance, and Capacitance to Analog Signals

Looking at a sensor interface system and the four types of AFE circuits that are differentiated according to their inputs, they all share the common feature of being feedback circuits² centered around an operational amplifier (Op-amp). As each of these circuits and their operations will be given a closer look, it will be important to understand that the input current, input voltage, input resistance, and input capacitor are the four types of output for a sensor device.

²Feedback circuit: a circuit that feeds back some of the output to the input of a system

Figure 3. The four types of analog front-end (AFE) circuits

Firstly, the current-input front-end—also called a transimpedance amplifier—is a circuit that uses an op-amp and a feedback resistor (R_feed) to convert the input current into a proportional output voltage. While the op-amp is an integrated circuit that can amplify weak electric signals, a R_feed controls the amount of output. As the gain, or the magnitude of an amplifier’s output and input, is determined by the size of the R_feed, the resistor requires stability.

The voltage-input front-end can be implemented by adding a gain resistor (R_gain) to the current-input front-end, and it amplifies to the desired gain and produces an output voltage. Consequently, balancing the proportion of R_feed and R_gain leads to the desired gain and allows for easy adjustments within the semiconductor while leading to minimal changes coming from the process or fluctuations in temperature. However, a voltage drop, or the amount of voltage loss that occurs through all or part of a circuit due to impedance, occurs when the input current flows entirely through the R_feed.

Next, the resistance-input front-end has a very similar structure to the voltage-input front-end. But there are also differences such as the input voltage being replaced with the reference voltage while the gain resistor is replaced with the input resistor for the parts to be operational. In the case of these front-ends, a stable reference voltage and R_feed are required to secure an accurate resistance-to-voltage gain. And to compensate for the inverse relationship between the output voltage and the input resistor, an inverse transformation is later performed on the digital output of the ADC to obtain an input resistor value.

Finally, a capacitance-to-voltage conversion is obtained through more intricate operations that are absent from the previously mentioned structures. The circuit uses a feedback capacitor instead of an R_feed while a switch is added to the input capacitor. The operation here follows a specific procedure. First, the reference power source and the input capacitor are connected through the switch to create an electrical charge. Then, this connection is discontinued while another switch is connected to the op-amp front-end to transfer the electrical charges to the feedback capacitor. As a result of this process, the output voltage is converted proportionally to the size of the input capacitor.

So, what impact does the AFE have on the performance of a sensor interface system? As the AFE is the first stage of the sensor interface system, the key requirements of having a wide input range, linearity, and a low noise level largely determine the performance of the sensor interface system. It is especially the low noise that has a significant impact as the noise of the feedback loop from the op-amp becomes the input-referred noise³ of the front-end. Thus, designing a low-noise op-amp is very critical. The main noises in an op-amp consists of thermal and flicker noise. The most crucial factor in reducing thermal noise is the semiconductor transistor that makes up the first input end of a circuit. In the first input end of an op-amp, voltage is converted to current by the trans-conductance⁴ of a transistor. Since the output current and the input voltage are linearly related as responding and control, respectively, arising problems can be compensated by maximizing the trans-conductance parameter. In some cases, flicker noises are even more emphasized than thermal noises. And as the power density of these flicker noises increases as the frequency decreases, reducing the flicker noise in the circuit that deals with low-frequency signals below tens of kHz is critical.

³Input-referred noise: The noise voltage or current that, when applied to the input of the noiseless circuit, generates the same output noise as the actual circuit does

⁴Trans-conductance: The electrical characteristic relating the current through the output of a device to the voltage across the input of a device

One way to solve this problem is to increase the physical size of the transistor in the first input end to reduce the fundamental flicker noise. In addition, techniques such as auto-zeroing and chopper stabilization—which reduces energy at the chopping frequency and maintains low noise levels at lower frequencies—can be used to remove noise more effectively. These techniques have been widely applied to many sensor interface semiconductor systems.

ADC: Low-power, High-resolution Connections for Sensor Interfaces

The voltage signals converted in the AFE go through an ADC to turn into digital signals, or the final output of a sensor interface. An ADC requires high resolution to accurately capture the analog data from the sensor device, and the retrieval of the data requires high energy efficiency.

Figure 4. The measurement of the power density and frequency of a N-bit CDAC

A successive-approximation-register ADC (SAR-ADC), which is one of the oldest ADC structures, possesses such high resolution and high energy efficiency on top of having an intuitive operation based on a circuit formation and binary search algorithm that is reminiscent of a digital logic. These attributes make it compatible with the latest semiconductor processes and, consequently, high in demand.

A SAR-ADC consists of an N-bit capacitor digital-to-analog converter (CDAC), a comparator, a SAR logic, and a track and hold (TAH). At its core, however, is the CDAC which uses a capacitor to convert a digital input into an analog output. At this time, the resolution of the SAR-ADC is inversely proportional to the number of bits in the CDAC, making the resolution become the smallest size of voltage the CDAC can exhibit. The number of bits in the CDAC also determines the magnitude of the quantization noise⁵. Quantization noise degrades resolution and appears uniformly over the entire frequency range of the power density.

Then, is it possible to increase the number of bits in the CDAC indefinitely to reduce the quantization noise and increase resolution? Although this is theoretically possible, there are practical limitations to implementing this in a semiconductor circuit. First, there are limitations within the CDAC as there are physical gaps between the capacitors carrying the bits in the CDAC due to constraints in the semiconductor process. The problem arises when the physical gap increases to cause errors in the form of harmonic distortion⁶ with linear degradation. This problem becomes even more severe as the size of the capacitor decreases. But there is even another limitation of the comparator that causes additional problems. As a comparator is responsible for comparing input voltages and outputting them, the size of the voltages that the comparator can compare is limited by various factors including thermal noise, flicker noise, and offset voltage⁷—even if the CDAC ideally produces a very small voltage. Finally, the resistance in the sampling switch at the input end increases the number of capacitors, and this causes greater thermal and KT/C noise⁸. In other words, it is physically challenging to improve resolution by significantly increasing the number of bits.

To overcome these limitations, the size of the gap between the capacitors can be minimized by adding or subtracting capacitors to each of the capacitors responsible for each bit in the CDAC. For the comparator, the limitations can be mitigated by using methods such as maximizing the trans-conductance parameter and frequency that was used for the AFE when addressing thermal noise and flicker noise. As the offset voltage has the same characteristics as flicker noise, it can also be removed in the same way. Finally, KT/C noise can be reduced below the desired noise level by increasing the size of the capacitors in the CDAC while increasing the sampling frequency. Using these techniques, low-power SAR-ADCs have been developed to obtain the level of 18-bit resolution.

⁵Quantization noise: The difference between the actual analog input that is typically represented by a sine wave and the value of the smallest discrete step or least significant bit

⁶Harmonic distortion: The distortion of the signals due to harmonic frequencies of a periodic voltage or current

⁷Offset voltage: The result of a difference in voltage between the outputs of two op-amps

⁸KT/C noise: The total thermal noise power added to a signal when a sample is taken on a capacitor

The Future of Next-generation Sensor Interface Technologies

The amount of physical, chemical, and biological data is growing at a staggering rate due to the advancement and expansion of today’s highlighted AI technology. Additionally, technologies that require real-time interaction such as AR and VR are being integrated into our daily lives while the use of mobile, wearable, and IoT/IoE devices is also on the rise. In line with these developments, the demand for related sensors is expected to grow exponentially, and the types and number of sensors required in a relevant system is anticipated to increase.

Based on this outlook of the future, it can be expected that developments in sensor interface systems will concentrate on technologies for miniaturization and low power consumption. Such goals were highlighted in recent papers presented at the International Solid-State Circuits Conference (ISSCC), one of the world’s most prestigious semiconductor conferences. The first type of trend discussed was the direct conversion sensor interface technology which converts data from sensor devices directly into digital values without using an AFE, which takes up a lot of power and space. Another trend that has a more direct approach to sensor solutions is a flexible type of technology that allows a single sensor interface system to be compatible with any of the voltage, current, resistance, and capacitance sensor devices. This minimizes the need for interfaces that correspond to various types of existing sensor devices. Finally, a third trend in the form of multi-modal sensor interface technology makes it possible to achieve simultaneous sensing with a single circuit system by complementing the flexible technology that has limitations in simultaneously sensing multiple sensor devices. These technologies are still in the early, conceptual stages, but they are expected to increasingly appear in the future.

Through this article, it was established that the main signals to be identified by sensing systems are resistance, capacitance, voltage and current. High-performing front-end circuits are required to convert these physical variables to an electrical signal that will further be processed by an ADC, with a common example being an SAR-ADC. To integrate more and more sensors into increasingly miniaturized devices, the sensing circuits need to offer ultra-low noise performance to achieve fine resolutions, maintain, linearity, and minimize power consumption as much as possible. This would lead to making sensor system design a challenging but exciting field. With increasing demands in digital connectivity and data acquisition from surrounding environments, numerous innovations are expected to emerge in sensor system design in the near future.

<Other articles from this series>

[DGIST Series] How the Quest for AI Led to Next-Generation Memory & Computing Processors

[DGIST Series] How Broadband Interface Circuits Are Evolving for Optimal Data Transfer

[DGIST Series] The Role of Semiconductor Technologies in Future Robotics

[DGIST Series] The Technologies Handling the Growing Data Demands in Healthcare

[DGIST Series] Silicon Photonics: Revolutionizing Data Transfers by Unleashing the Power of Light

[DGIST Series] AI-Powered Micro/Nanorobots to Revolutionize Medical Field

The post [DGIST Series] Sensor Interfaces and ADC Circuits: Bridging the Physical and Digital Worlds first appeared on SK hynix Newsroom.

Figure 1. The poster for the 1966 sci-fi film Fantastic Voyage (a) and magnetic microrobots for 3D cell culture and targeted transportation (b) (Figure 1b image credit: Wiley-VCH)

While the idea of shrinking biological entities remains far-fetched, reductions in the sizes of non-living things are now possible. Advances in micro-electromechanical systems (MEMS) and semiconductor technologies have enabled the micro/nanoscale fabrication of robots and devices, thereby revolutionizing micro/nanorobotics. Today, micro/nanorobots are used in biomedical applications, such as precise drug and cell delivery, non-invasive diagnosis, and targeted therapies.

In the latest article in our series from DGIST faculty which covers slightly different topics from the previous semiconductor-related posts, Professor Hong-soo Choi from the Department of Robotics Engineering explains how the application of AI has helped to advance the use of micro/nanobots in the medical field.

Realizing Untethered Microrobots Through Optical, Acoustic and Magnetic Actuation

Conventional robots require integrated circuits and power to actuate their motors, making fabrication difficult at micro/nanoscales. Actuation by optical, acoustic, or magnetic sources solves both of these issues by allowing wireless actuation of these robots.

Figure 2. Optical (a), acoustic (b), and magnetic (c) actuation schemes for microrobots

A. Optical-based Actuation

Optical-based actuation schemes use changes in the physical properties of a specific material upon exposure to a light source. Liquid crystal elastomers¹ (LCE) combine the elasticity of a polymer with the self-organization of the liquid crystalline phase to create flexible structures that undergo reversible deformation. Actuators based on LCE and LCE-based composites can be actuated using a light source, as shown in Figure 2a. Several actuation schemes and fabrication techniques for these materials have been presented, which allow photothermal and photochemical actuation of liquid crystal elastomer-based soft robots. The actuation of soft robots depends on the light source and its interaction with the environment. Considering the progress in liquid crystal elastomers, this actuation had predominantly become a design issue which was solved using biomimetics². Various bio-inspired designs for liquid crystal elastomers in soft miniature robotics have been presented, such as artificial cilia³ and caterpillar-inspired devices.

¹Elastomer: A polymer with viscosity and elasticity such as natural rubber.
²Biomimetics: The study of principles of nature which can be applied to materials, synthetic systems and machines that can imitate the structure and function of native biological systems.
³Cilia: Hair-like structures that extend from the cell body into the fluid surrounding the cell.

Another photothermal mechanism is one which uses a metal layer on the microrobot and propels the device with the heat generated from a laser. Additionally, a control scheme has been presented where optoelectronic tweezers, which use projected optical images to manipulate tiny particles, can actuate the microrobot.

B. Acoustic-based Actuation

Acoustic-based actuation schemes rely on sound waves to propel micro/nanorobots. Similar to optical actuation, acoustic-based actuation depends on the robot design. However, the choice of materials gives it an advantage in terms of allowing diverse bio-compatible designs. Two-photon polymerization and photolithography⁴ are widely used for the fabrication of such microrobots. Three major acoustic actuation schemes are bubble propulsion, flexible-tailed propulsion, and in situ⁵ micro-rotors.

⁴Photolithography: A technique that uses light to produce minutely patterned thin films of suitable materials over a substrate, such as a silicon wafer, to protect the selected areas.
⁵In situ: A Latin phrase meaning on-site or in its original place.

In a bubble propulsion scheme, a bubble trapped inside an open-ended structure vibrates along with the applied acoustic field as shown in Figure 2b. The two ends of the structure allow the intake and discharge of fluid, and the propulsion increases as the applied frequency approaches the resonance frequency of the bubble. Flexible-tailed propulsion is inspired by single-celled organisms and flagella⁶. It uses flexible-tailed microrobots to generate counter-rotating eddies⁷ for propulsion. Meanwhile, in situ micro-rotors with a fixed axis of rotation have a working principle similar to flexible-tailed propulsion.

⁶Flagella: Microscopic hair-like structures involved in the locomotion of a cell.
⁷Eddy: The swirling of a fluid and the reverse current created when the fluid flows past an obstacle.

C. Magnetic-based Actuation

Magnetic-based actuation schemes use systems based on electromagnets, permanent magnets, or both. In contrast to acoustic and optical-based actuation, magnetic actuation requires an emphasis on control theory and microrobot structure. As shown in Figure 2c, the three main control strategies for the magnetic actuation of microrobots involve gradient, rotating, or oscillating magnetic fields.

Under gradient magnetic fields, microrobots experience magnetic torque and magnetic force. In this case, field vectors guide orientation and the magnetic field gradient controls motion. Any structure with magnetic properties can be guided in this manner. In contrast, a rotating field is generated by the rotation of the magnetic field vector around an axis. Under a rotating field, helical microrobots propel themselves forward using fluid forces; cylindrical microrobots can use a tumbling motion; and spherical microrobots can roll on a surface. Finally, oscillating fields are generated by the movement of the field vector up and down in a plane. Flexible or tailed microrobots and soft robots are commonly actuated with this strategy.

The Rise of AI and Machine Learning

In recent years, AI has undergone a rapid transformation and is now utilized in a range of sectors including transportation, banking, healthcare, and industrial automation. The rise of AI was triggered by state-of-the-art research, advances in semiconductor technology, and the availability of extensive data.

Figure 3. The processes of supervised learning, unsupervised learning, and reinforcement learning

Machine learning (ML) encompasses a subset of AI processes in which a machine learns to perform a specific task through analysis of a provided dataset. As shown in Figure 3, there are three major ML algorithms: supervised learning, unsupervised learning and reinforcement learning. ML combined with a deep neural network (comprising multiple artificial neural networks layered on each other) is known as deep learning. Deep learning has enabled ML to manage information and make human-level or even better decisions.

Supervised learning algorithms require a labeled dataset from which the machine can learn and then infer based on some input. Considering the extensive data currently available, supervised learning has become a very powerful tool for classification, regression, and forecasting tasks. Object recognition, speech recognition, and stock market prediction are among the practical uses of supervised learning.

Unsupervised learning algorithms are used when a dataset is not labeled. These algorithms are more complex than supervised learning as they must comprehend data without relying on human knowledge in the form of labels. Unsupervised learning is used for tasks such as clustering, detecting similarities and anomalies, visualizing data, and labeling unlabeled data.

Reinforcement learning (RL) is the nearest of the three to the human learning process. The algorithm learns to complete a specific task by exploring a particular scenario or environment and learning from its mistakes. The dataset is generated at runtime while the algorithm performs some action in the environment. The RL algorithm can learn without any human interference and is generally used to enable machines to understand tasks humans are already performing, such as playing computer games, driving cars, and trading stocks. Notably, in 2016 an RL-powered computer program named AlphaGo defeated professional Go world champion Lee Sedol.

Micro/Nanorobots and AI

In the past decade, AI has become increasingly valuable in robotics. It has allowed control of robots, followed by vision and then autonomous control, thus providing a new perspective in robotics. Considering the recent demand for autonomous vehicles and the increase in autonomous robots in everyday life, the idea of humanoids replacing humans appears feasible in the not-too-distant future. In the early 2000s, the Da Vinci surgical system allowed robotics to become a component of conventional surgery. It allows surgeons to perform invasive and complex procedures with relative ease, particularly in small, confined spaces.

Complex tasks for machines become considerably more difficult when they are conducted on a micro- or nanoscale inside the human body. Accordingly, researchers have increasingly incorporated AI in microrobotics over the past decade. Challenges in terms of control/actuation, automation, imaging, and design optimization at the micro/nanoscale are being addressed with AI.

Micro/nanorobot design optimization is important because of its central role in the efficiency of control schemes. ML is widely used as a design optimization technique in areas such as the automotive industry, antenna design, and composite material design. In comparison to conventional optimization techniques, ML can substantially accelerate the optimization problem and provide better designs. A recent example of ML-based optimization at the micro/nanoscale is the optimal adhesive fibril design reported by Donghoon Son et al. in 2021, in which their ML approach outperformed proposed designs by 77%.

Imaging at a micro/nanoscale inside the human body is challenging because optical imaging techniques cannot be used. Magnetic resonance imaging, ultrasound, and X-rays are used to detect the positions of micro/nanostructures both in vivo⁸ and ex vivo⁹. In 2022, a study by Karim Botros et al. on autonomous detection and tracking based on ultrasound imaging of a microrobot found they were able to track the robot with an accuracy of more than 90% using deep learning. In the same year, Mehmet Efe Tiryaki et al. presented deep learning-based three-dimensional tracking of a magnetic microrobot using two-dimensional magnetic resonance images and were able to achieve an accuracy of 87.5% in vitro.

⁸In vivo: A Latin phrase meaning inside a living organism.
⁹Ex vivo: A Latin phrase meaning outside a living organism.

Control and actuation at a micro/nanoscale inside the human body have greater complexity because of Brownian motion¹⁰ interactions between the micro/nanorobot’s structure and the environment, and the effect of the actuation mechanism on the robot. These complexities hinder open-loop control of the robots, making conventional controllers like proportional-integral-derivative (PID)¹¹ operations unreliable. RL has therefore become a reliable ML technique for control actuation schemes because it is less dependent on human input and can generate the data itself.

¹⁰Brownian motion: The random movement displayed by small particles that are suspended in fluids.
¹¹Proportional-integral-derivative: A control loop feedback mechanism widely used in industrial control systems which uses feedback to continuously adjust the output of a process or system to match a desired setpoint.

In 2018, Santiago Muiños-Landin et al. published one of the first works regarding the use of RL with a microrobot. They navigated a light-controlled artificial microswimmer, a microscopic device able to move in fluids, under the influence of Brownian motion. They divided the workspace into a grid-like structure and used the RL to identify the optimal path for the microswimmer to reach a target in the grid. For magnetic actuation, in 2021 Michael R. Behrens and Warren C. Ruder presented a smart helical-shaped microrobot with an RL-guided time-varying magnetic field for optimal motion inside a circular fluidic channel. The most recent work regarding an acoustic actuation scheme and RL was published by Matthijs Schrage et al. in 2022 in which they showed programmable control of a swarm of microrobots via ultrasound. Moreover, Lidong Yang et al. showcased fully autonomous navigation of magnetic nanoparticles via RL in 2022. Their RL approach could control the shape and trajectory of a nanoparticle swarm for optimal navigation under time-varying magnetic fields.

Future of AI-Powered Microrobotics

Figure 4. Neural reconstruction via magnetically actuated microrobots on a microelectrode array

Recent works focused on microrobots in biomedical applications, such as neural reconstruction as shown in Figure 4 have led to an increasing need for in vivo and human trials. Current work in microrobotics and AI suggests there is potential for unlimited progress. Combinations of micro/nanorobots in biomedical fields with AI for very complex tasks should soon lead to in vivo testing and clinical trials as AI can eliminate human error and limitations at the micro/nanoscale.

Considering the significant advancements thus far in the performance of microscale imaging, localization, control, and fabrication, AI-powered micro/nanorobots will almost inevitably replace humans in operating theaters in the future. Along with these technologies, the use of semiconductors will also increase as machine learning develops further in the future.

<Other articles from this series>

[DGIST Series] How the Quest for AI Led to Next-Generation Memory & Computing Processors

[DGIST Series] How Broadband Interface Circuits Are Evolving for Optimal Data Transfer

[DGIST Series] The Role of Semiconductor Technologies in Future Robotics

[DGIST Series] The Technologies Handling the Growing Data Demands in Healthcare

[DGIST Series] Silicon Photonics: Revolutionizing Data Transfers by Unleashing the Power of Light

[DGIST Series] Sensor Interfaces and ADC Circuits: Bridging the Physical and Digital Worlds

The post [DGIST Series] AI-Powered Micro/Nanorobots to Revolutionize Medical Field first appeared on SK hynix Newsroom.

¹Photonics: The study of matters related to photons, which are the fundamental units of light.

Figure 1. Conceptual diagram of silicon photonics technology (Image credit: PhotonHub)

In this latest article in our series by faculty from South Korean university DGIST, Professor Sangyoon Han of the Robotics and Mechatronics Engineering department explains silicon photonics, its use in data centers and optical links, the manufacturing process of silicon photonics chips, and the technology’s future applications.

The Significance of Silicon Photonics in Data Centers

Although data centers and their significant role in operating a host of technologies may be a familiar concept to many people, it is less widely known that silicon photonics technology is essential to the running of these centers. Tens of thousands of servers in a data center are connected not by wires but by optical fibers that act as pathways for light, enabling high-bandwidth communication. Thus, the electrical signals generated in the servers need to be converted into light, or the conversion needs to happen the other way around in order for light and electrical signals to pass through the optical fibers. To facilitate this, optical transceivers—devices developed using silicon photonics technology—are applied.

Figure 2. A prototype of a silicon photonics optical transceiver (Image credit: IMEC)

Able to handle both light and electrical signals simultaneously on a single chip, silicon photonics technology is considered an ideal platform to form optical transceivers. Due to light’s low attenuation² and their parallel processing capability, optical transceivers enable communication between servers at speeds of up to 400 Gbps (gigabits per second). Considering that electrical wires can only achieve speeds of single-digit Gbps, it is clear that optical transceivers are essential for the efficient and speedy operation of data centers. Subsequently, around a million optical transceivers are used in a single large-scale data center.

²Attenuation: In the optical field, it is the rate at which the signal light decreases in intensity.

Use of Silicon Photonics in Optical Links

The application of silicon photonics spreads beyond data centers as it is currently being used in other areas which require high-bandwidth data transmission. Recently, there has been a growing trend in using optical waveguides instead of copper wires to connect chips that are located within a few centimeters of each other. This is where silicon photonics play a key role as there is active research into the use of silicon photonics-based optical links³ for interconnection between GPUs and between the cores in a multi-core CPU. These optical links are also being considered to connect CPUs and memory chips. As a prime example of such applications, Silicon Valley chip startup Ayar Labs is working with NVIDIA to develop technology that could embed optical links into a large-scale GPU system as shown in Figure 3.

³Optical link: An optical transmission channel designed to connect two end terminals or to be connected in series with other channels.

Figure 3. Example of applying optical links in chip-to-chip technology (Image credit: Ayar Labs)

How Silicon Photonics Chips Work

So, how exactly does a silicon photonics chip work? To make it easier to understand, it is helpful to compare the technology with electronic circuits. Although electronic circuits look complex, they are mostly made up of transistors and copper wires that connect the transistors. The functions of transistors and wires in an electronic circuit are parallel to the roles of optical modulators and waveguides in a silicon photonics chip, while the power supply of an electronic circuit corresponds to the role of a laser for such chips. Additionally, a photodetector is used to convert optical signals into electrical signals in a silicon photonics chip.

Figure 4. Conceptual diagram of the process of an optical modulator

Just as wires carry electrons, waveguides carry photons, or light. They work as channels that transmit light without any loss, similar to optical fibers. For silicon photonics chips, they use ultra-thin waveguides of less than 1 micrometer⁴(μm) in diameter that are manufactured using semiconductor processes. Furthermore, optical modulators alter the transmissivity of the waveguides to change the intensity of the light passing through the waveguides—this generates optical signals as shown in Figure 4.

⁴Micrometer (μm): 1 micrometer is one-millionth of a meter.

The Manufacturing Process of Silicon Photonics Chips

Figure 5. Silicon photonics chips fabricated on a 12-inch wafer (Image source: Nature)

The process of manufacturing silicon photonics chips is very similar to making electronic circuits through the CMOS process. Both processes use silicon as a base material, and for silicon photonics chips it is advantageous to be manufactured through the CMOS process rather than developing a whole new manufacturing process in terms of time, cost, and efficiency. In turn, the similarity of the processes makes it straightforward to produce silicon photonics chips on existing semiconductor production lines.

Moreover, the size of silicon photonics devices, which measure only a few micrometers, is optimal for production as they can be easily manufactured using nanometer⁵ processes. As a result, despite the relatively short history of silicon photonics, major foundries around the world have started to produce silicon photonics chips on 12-inch wafers. With more of these world-renowned foundries entering the silicon photonics business, it is foreseeable that the silicon photonics market will grow significantly going forward.

⁵Nanometer (nm): 1 nanometer is one-billionth of a meter. Therefore, 1μm = 1,000 nm.

Silicon Photonics in Autonomous Driving Sensors

Silicon photonics has not only made a significant impact in data transmission—an area where light holds an advantage—but is also being applied to a growing number of fields these days. For example, the technology is being utilized to achieve higher performance and miniaturization of autonomous driving sensors such as Light Detection and Ranging (LiDAR). Most LiDAR systems available in the market today are difficult to mass-produce at low costs because they are made by manually assembling components such as motors and lenses. But silicon photonics technology is expected to become a new solution to this as it enables these LiDAR systems to be manufactured with improved performance, energy efficiency, and lower costs. In practice, this technology will ultimately lead to a drastic fall in costs when adding an autonomous driving system to a car.

Future Applications of Silicon Photonics

Silicon photonics is also making it possible to develop new computing technologies beyond existing paradigms. Such next-generation computing technologies include: AI processors that use light’s ability of parallel processing to compute multiple AI inferences with a single physical device; quantum computing that transcends the limits of classical physics; and quantum cryptography communications that are physically impossible to wiretap.

With the emergence of advanced technologies such as large-scale AI models, the demand for improved computing power and data processing capabilities from hardware is greater than ever. As a result, the existing paradigm of electronic semiconductors will need to evolve to keep up with this strong demand, and this evolution can be materialized with silicon photonics technology. As the technology harnesses the power of light in semiconductors, physical limitations can ultimately be overcome so semiconductors can realize previously unobtainable applications. Looking ahead, it is also anticipated that the use of silicon photonics will result in colossal progress in computing and AI applications.

<Other articles from this series>

[DGIST Series] How the Quest for AI Led to Next-Generation Memory & Computing Processors

[DGIST Series] How Broadband Interface Circuits Are Evolving for Optimal Data Transfer

[DGIST Series] The Role of Semiconductor Technologies in Future Robotics

[DGIST Series] The Technologies Handling the Growing Data Demands in Healthcare

[DGIST Series] AI-Powered Micro/Nanorobots to Revolutionize Medical Field

[DGIST Series] Sensor Interfaces and ADC Circuits: Bridging the Physical and Digital Worlds

The post [DGIST Series] Silicon Photonics: Revolutionizing Data Transfers by Unleashing the Power of Light first appeared on SK hynix Newsroom.

Consequently, there is heightened interest in taking care of one’s health and the development of medical technologies to alleviate the demands on healthcare networks. These technologies include personalized health monitoring systems like point-of-care ultrasound devices that can constantly observe individual health statuses and AI systems such as 3D ultrasound imaging which can analyze and predict risk factors so people can rapidly receive treatments. Such advancements have increased the amount of data that needs to be processed, leading to increased demands in memory and, also, semiconductor memories that are used in data centers.

In this latest article in our series from DGIST faculty, Professor Jaesok Yu from the Department of Robotics and Mechatronics Engineering explains the need for solutions such as semiconductor technologies that can manage the data bandwidth of advanced medical systems, including 3D ultrasound imaging and AI diagnostic tools.

Advancement of Ultrasound Imaging Technology

Figure 1. The evolution of ultrasound imaging systems

Ultrasound is the most suitable medical imaging technology for point-of-care testing¹. Its affordability, safety, and ability to be miniaturized enable people to regularly use ultrasound equipment at home. Other medical imaging technologies such as magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET) which are commonly used in clinical practice are difficult to miniaturize. Moreover, CT and PET also use radiation which makes them a safety risk for home use. In contrast, ultrasound imaging technology can not only be miniaturized but is also comparatively safe as it is a non-radiological alternative. In addition, as an ultrasound provides a real-time view inside a patient’s body, it is particularly suitable for monitoring vascular diseases such as strokes that require rapid diagnoses.

¹ Point-of-care testing: A medical test carried out near or in the presence of the patient and without the need to send a sample to a laboratory.

Recent research and development in ultrasound imaging technology has been divided between premium, high-performance systems used in hospitals and point-of-care systems which are portable and used in a variety of settings. In particular, the market for miniaturized devices has grown significantly in recent years and has gained even more attention since the start of the COVID-19 pandemic. For instance, an American startup developed a miniaturized point-of-care ultrasound imaging system called Butterfly IQ which can connect to a smartphone.

Supporting the Data Bandwidth of State-of-the-art Ultrasound Imaging

Cutting-edge, ultrasound imaging technologies such as volumetric imaging or ultrafast imaging technologies increasingly demand a significant amount of data. This trend in data demand is caused by three-dimensional imaging that is required for enhanced personalized care.

Today, the most significant limitations of ultrasound systems are that they can only display 2D cross-sectional images and the results can vary greatly based on the operator’s proficiency—known as operator dependency. Consequently, there has been research on developing 3D imaging technology that would reduce operator dependency as much as possible. Although a limited form of 3D ultrasound imaging technology can now be implemented, a number of technical challenges makes it difficult to achieve high-quality 3D images in real-time.

One of the main obstacles is data bandwidth. For portable ultrasound systems to produce 3D images, the number of elements in an ultrasound array transducer² needs to be increased from an ‘n’ number of one-dimensional (1D) linear arrays to an ‘n^2’ number of 2D planar arrays. Given that n, or the number of 1D linear arrays, is typically between 128 and 256, it is clear that 3D imaging requires significantly more data to be processed. Typical 2D ultrasound imaging systems, which receive data by connecting an analog-to-digital converter (ADC, typically 40~60 MHz) to each element, have a maximum data bandwidth of just several gigabytes per second. It can therefore be roughly estimated that 3D ultrasound systems would require bandwidths that reach hundreds of gigabytes per second.

² Transducers: A device that uses piezoelectric materials to convert sound waves into electrical signals and vice-versa. It transmits and receives waves simultaneously.

Figure 2. A comparison of 2D and 3D ultrasound imaging systems

Therefore, numerous techniques that provide 3D imaging while reducing data usage are being studied including sparse array imaging, compressive sensing, and deep learning-based image reconstruction. There are also a number of studies on various techniques aimed at fundamentally improving bandwidth. It is, therefore, critical to overcome the limitations of 3D ultrasound imaging by developing technologies that can efficiently transmit and process the exploding amount of data.

The Potential and Challenges Facing AI in the Medical Field

Once normalized 3D image data is obtained, it needs to be interpreted in conjunction with relevant monitoring data and be ready to be used immediately for early diagnosis and prediction. Currently, medical professionals take on these duties, which are time-consuming and costly. In light of this, it is anticipated that AI will partially replace or assist these doctors as the technology evolves. However, this integration of AI into the medical field is unlikely to be smooth. Although AI has produced remarkable results in many areas, it faces significant challenges to become widely used in the medical field.

Among the various reasons for the shortcomings of AI medical diagnoses, the main factors were the complexity of data related to certain diseases and the subjectivity in interpreting this data. In most cases, the establishment of well-refined training datasets determines how well AI performs in the healthcare sector. However, in general only highly trained doctors are able to select, label, and create these datasets. This proved to be the biggest challenge in developing AI for healthcare. Additionally, on top of the limited amount of medical data that exists, it also takes an enormous amount of time and costs to refine the data. This, in turn, makes it difficult to obtain high-quality datasets. Even if such datasets are obtained, the data may be interpreted differently as there is a degree of subjectivity when a person generates the training datasets.

An additional challenge is synchronizing the massive amount of patient background data such as race, nationality, and culture that is collected to train the diagnostic medical AI. This seems to be the main reason why the diagnosis alignment rates of medical AI systems vary depending on the country of use.  There was also the hassle of having to manually enter all of the electronic medical record (EMR) information into the system, which was very inconvenient and made it difficult to learn new information. This issue of securing data in the medical field has led to the development of technologies such as unsupervised learning that do not require reference data but only need large amounts of data to learn. Recent studies have shown that these technologies are attracting much attention by offering performance comparable to supervised learning³ systems.

³ Supervised learning: A subcategory of machine learning and AI which uses labeled datasets to train algorithms to classify data or predict outcomes accurately.

The Essence of Data in Medical AI

As it was established that the interpretation and synchronization of data presents issues for AI’s integration into healthcare, improving the collection and utilization of data will be key to advancing AI in the medical field. In the U.S., the importance of healthcare data has been recognized, and multi-institutional collaborations are underway for the collection of medical data. A prime example of such a collaboration between industry, academia, and hospitals is the Pittsburgh Health Data Alliance. Consisting of Carnegie Mellon University, the University of Pittsburgh, and the University of Pittsburgh Medical Center, the alliance collects healthcare data in the region to support research related to AI in the medical field.

Figure 3. The process of collecting and processing data for AI medical systems

As data is growing exponentially in both volume and importance, the most critical aspects of an AI system are efficient computation and the ability to process large amounts of data and bandwidth. In fact, the main reason for AI being in the spotlight today is due to the fact that it is backed by huge amounts of data and hardware technology that can handle this data. However, this was not always the case as AI has faced major challenges in its path to mainstream adoption.

Although AI’s fundamental principles were first proposed in the 1940s, the technology did not perform as expected due to limited hardware resources and minimal data available, causing the development of neural networks to stall. However, around 70 years later a convolutional neural network (CNN) called AlexNet demonstrated the incredible potential of AI. Backed by deep neural networks and big data, AlexNet showcased its capabilities in the 2012 ImageNet Large Scale Visual Recognition Challenge (ILSVRC), which evaluates algorithms for object detection and image classification. The system’s most significant feature was its ability to learn from training data.

In the past, machine learning with large volumes of data was challenging due to difficulties in data collection, limited computing power, and bandwidth. However, the development of parallel computing and high-bandwidth hardware made it possible to process huge amounts of data and train complex neural networks. Not only has the amount of data collected increased enormously, but techniques such as data augmentation have greatly increased the amount of training data by reproducing existing data to improve performance. Indeed, AlexNet has 60 million network parameters and uses two GPUs to perform massive operations efficiently. These networks are getting bigger as evident in the ultra-large artificial neural networks such as ChatGPT-3 that have as many as 175 billion parameters, nearly a 3,000-fold increase in size over the past decade. As the size of networks and training data increases, so does the need for hardware that can handle them efficiently.

Semiconductor Technologies’ Role in Next-Generation Healthcare

In the future, healthcare technology is expected to become personalized, allowing for constant monitoring of an individual’s health and early diagnosis of diseases. This will lead to an exponential increase in the amount of data while the importance of data will also grow, especially when combined with AI. As a result, the future of healthcare technology depends on the ability to collect and efficiently process well-refined data. Hardware, especially innovative semiconductor technology, is essential in this process. However, as current semiconductor memory technologies have reached the stage where nanometer (nm) processes are possible, performance improvements are beginning to reach their limits.

Consequently, global semiconductor companies are researching and investing in innovative next-generation semiconductors to meet this challenge. Depending on the data requirements of the respective systems, different types of semiconductor memories are used. Most systems such as point-of-care typically use Double Data Rate (DDR) 2 DRAM through DDR4 DRAM, while systems that process heavy loads of computation with GPUs use Graphics Double Data Rate 5. Additionally, systems that have extremely large data such as data centers for cloud computing use High Bandwidth Memory 3 (HBM3) or memory with built-in PIM-based accelerators. The former has rapid operating speeds and high bandwidth, enabling faster data processing for AI applications, and the latter allows computation to be performed directly in memory. These features make both technologies a good fit for training diagnostic medical AI down the road.

Need for a Paradigm Shift in Healthcare

To manage the growing data in the healthcare field, there needs to be further research into semiconductor technologies beyond transistors, and it will become critical to develop innovative solutions that can fundamentally change the current paradigm. Such game-changing advancements could include accelerators based on silicon photonics that use light for computing and data transmission. As the need for a new age of personalized healthcare grows with the aging of societies around the world, advancements in AI and data collection will prove to be key to realizing the next generation of healthcare.

<Other articles from this series>

[DGIST Series] How the Quest for AI Led to Next-Generation Memory & Computing Processors

[DGIST Series] How Broadband Interface Circuits Are Evolving for Optimal Data Transfer

[DGIST Series] The Role of Semiconductor Technologies in Future Robotics

[DGIST Series] Silicon Photonics: Revolutionizing Data Transfers by Unleashing the Power of Light

[DGIST Series] AI-Powered Micro/Nanorobots to Revolutionize Medical Field

[DGIST Series] Sensor Interfaces and ADC Circuits: Bridging the Physical and Digital Worlds

The post [DGIST Series] The Technologies Handling the Growing Data Demands in Healthcare first appeared on SK hynix Newsroom.

In the third episode of our series from Daegu Gyeongbuk Institute of Science and Technology (DGIST) faculty, Professor Hoe-joon Kim from the Department of Robotics and Mechatronics Engineering will explain the future of robotics and the semiconductor technologies driving their development.

From Increased Autonomy to Energy Efficiency: The Future of Robotics

Figure 1. The six developments of future robotics

Robotic technologies are now utilized in various industrial sectors and the demand for more advanced robots is higher than ever. As these technologies evolve over the coming years, they will be defined by several key properties.

One of the most significant developments in future robotics will be the increase in autonomy. Currently, most robots require human supervision or programming to perform their tasks. However, robots in the future are expected to become more autonomous, able to learn from their environment, and make decisions independently. Such self-thinking robots will be made possible by integrating advanced technologies such as artificial intelligence (AI), machine learning, and computer vision¹.

¹ Computer vision: A field of AI that enables computers and systems to derive meaningful information from digital images, videos and other visual inputs, and take actions or make recommendations based on that information.

In addition to autonomy, versatility will also be a key trait of next-generation robotics. Robots are being developed to perform a wider range of tasks in various industries, such as manufacturing, healthcare, and sports and entertainment. Future robots must therefore be designed to adapt to different environments and situations, with modular designs that allow for easy customization and reconfiguration.

As robots are set to be used in a wider range of industries and increasingly work alongside humans, safety becomes an important consideration. In the future, robots will be designed with advanced safety features that minimize the risk of injury or damage. Many safety features, such as collision detection, force sensing, and alarm systems, can already be found in commercial robots.

As robots play an ever-growing role in our daily lives, it is becoming important to design robots that can interact with humans naturally and intuitively. Advanced human-robot interaction (HRI) technologies include natural language processing and speech recognition, as well as advanced sensors and actuators that can enable robots to mimic human movement.

In addition to interacting with humans, robots will also need to communicate with each other and other devices. Therefore, advanced wireless communication technologies will be required that can support large numbers of devices with low latency and high reliability.

Energy efficiency is also a major concern for many robotics applications, particularly those requiring robots to operate for long periods without recharging. In light of this, robots are being developed with more efficient power systems, such as batteries and fuel cells, and energy harvesting technologies that can convert ambient energy² into usable power.

²Ambient energy: Power sources in the environment such as radio waves, kinetic energy and solar power.

All of the aforementioned properties are crucial for the future evolution of robotics. In order to realize these features, advanced semiconductor technologies for sensing (sensors), moving (actuators), and thinking (processors) must be integrated into the robots of tomorrow.

The Importance of Semiconductor Technologies for Future Robotics

As robots become smarter, faster, and more adaptable to the environment, there is a greater need to integrate efficient and powerful components capable of sensing, actuating, and data processing. These components heavily rely on semiconductor technologies to help realize the potential of future robotics.

Processors: How Will the “Robots’ Brain” Spur the Evolution of Robotics?

Figure 2. Processors and other technologies which enable robots to “think”

One key semiconductor technology which will facilitate the development of robotics is the AI processor, a specialized semiconductor designed to accelerate the processing of AI algorithms. In robotics, AI processors will enable robots to make complex decisions and interact with their environment in real time. This technology will be crucial for autonomous robots that must make decisions based on real-time data from their sensors.

In addition to making decisions, robots in the years to come will utilize neuromorphic computing to become more human-like. Neuromorphic computing is an emerging semiconductor technology designed to mimic the structure and function of the human brain. This technology is particularly promising for robotics because it can enable robots to learn and adapt to new situations more quickly and effectively than traditional computing approaches. As it continues to advance, we can expect to see the development of robots that can perform increasingly complex tasks in a wider range of environments.

As robots are set to be used in even more industries going forward, they will need to use high-performance computing technologies such as GPUs and field-programmable gate arrays (FPGAs)³ to process and analyze large amounts of data in real-time. These technologies are particularly important for applications such as image and speech recognition, which require large amounts of computational power. In addition, edge computing⁴ will enable robots to operate in environments with limited connectivity or where real-time communication with a remote server is impossible.

³Field-programmable gate arrays (FGPA): A semiconductor device which can be configured to the desired application or functionality after it is manufactured.

⁴Edge computing: Allows devices in remote locations to process data at the “edge” of the network, either by the device or a local server. In robotics, edge computing enables data processing locally on the robot rather than in the cloud.

Enhanced communication capability will also be vital for future robotics. Therefore, it will be necessary to fully harness rapid 5G wireless technology to enable robots to operate and communicate in real time. This will make robots more responsive and capable of working together on various tasks in complex and dynamic environments.

Robotics often require a significant amount of power. As robots become more advanced, they will require more sophisticated power management technologies to ensure they can operate for an extended time. Technologies like energy harvesting, advanced batteries, and wireless charging will enable robots to operate without needing to be recharged or physically connected to an external power source.

Sensors: Will Future Robots be Able to Sense the World as We Do?

Advanced semiconductor technology will not only improve robots’ decision-making and communication capabilities, but also their ability to sense their environment and perform actions through sensors.

Figure 3. The various potential senses of future robots

CMOS image sensors (CIS) are already widely used in cameras and other imaging applications. In robotics, CIS will enable robots to “see” and interpret their environment. These low-power sensors are lightweight and capable of capturing high-quality images, making them ideal for robotics applications. SK hynix is developing time-of-flight⁵ (TOF) CIS technology, which can revolutionize how future robots perceive an object.

⁵Time-of-flight (TOF): The measurement of the time taken by an object, particle or wave to travel a distance through a medium. TOF sensors measure the time it takes for a light signal to travel from the sensor to an object and back.

Another type of sensor which can assist with environment perception is Light Detection and Ranging (LiDAR). These sensors, which use lasers to create a 3D map of the surroundings, are already utilized in autonomous vehicles and will likely become increasingly important for robots. LiDAR sensors will enable robots to navigate complex environments and avoid obstacles in real-time.

Sensors can also be used for safety purposes as they can provide environmental-based warnings. For example, gas sensors are semiconductor-based sensors capable of detecting the presence of specific gases. In robotics, gas sensors will enable robots to detect and respond to changes in their environment, such as the presence of toxic gases or other hazardous materials. In addition to toxic gases, various chemical sensors could be applied to robots. Recently, fine dust sensor-equipped drones have been commercialized to monitor air pollution.

As well as detecting the environment, sensors also need to identify the robot’s movements around its surroundings. Micro-electromechanical systems (MEMS) sensors are miniature sensors that detect a wide range of physical parameters, including acceleration, rotation, and pressure. They are already used in various applications, including smartphones, wearables, and automotive systems. In robotics, MEMS sensors will enable robots to detect their orientation, movement, and other physical parameters.

Actuators: How Will Robots of the Future Move?

While sensors allow robots to perceive their environment, actuators allow them to interact with the world around them.

Figure 4. Types of actuators which enable robots to move

In contrast to the conventional robots of today, future robots will be equipped with a softer, more delicate, and more efficient actuation scheme. These soft robots, which are designed to mimic the movement and flexibility of natural organisms, will feature flexible artificial muscles which could be made from electroactive polymers (EAPs). EAPs, which change shape in response to an electrical stimulus, are the ideal material for these “muscles” as they can create lightweight, flexible actuators capable of performing a wide range of movements. In addition, shape memory alloys (SMAs) can change form in response to a temperature change or electrical current. They are already used in some robotic applications, such as grippers and actuators for space exploration, and are likely to become more common as robots evolve.

Piezoelectric actuators, which convert electrical energy directly into linear motion, are already commonly used in robotics but are likely to become more widespread in the future. These actuators are lightweight, small, and precise, making them ideal for applications such as micro-robotics and medical devices. Recent advances in high-performance piezoelectric thin film deposition technologies will further promote their adoption. Similarly, micro-electromechanical systems (MEMS) actuators are miniature actuators capable of performing precise movements. They are already used in a wide range of devices, including sensors and switches, and are likely to become more commonplace in robotics.

The Key to the Future of Robotics

Semiconductor technologies are clearly set to play a critical role in enabling the capabilities of future robotics sensors. By combining these technologies with advances in AI, actuators, sensors, and other areas, we can expect to see robots become more advanced, capable, and integrated into our daily lives. For SK hynix, it is well positioned to be a global leader in semiconductor products for future robotics thanks to its top-level resources for the design, manufacturing, and systemization of the components.

<Other articles from this series>

[DGIST Series] How the Quest for AI Led to Next-Generation Memory & Computing Processors

[DGIST Series] How Broadband Interface Circuits Are Evolving for Optimal Data Transfer

[DGIST Series] The Technologies Handling the Growing Data Demands in Healthcare

[DGIST Series] Silicon Photonics: Revolutionizing Data Transfers by Unleashing the Power of Light

[DGIST Series] AI-Powered Micro/Nanorobots to Revolutionize Medical Field

[DGIST Series] Sensor Interfaces and ADC Circuits: Bridging the Physical and Digital Worlds

The post [DGIST Series] The Role of Semiconductor Technologies in Future Robotics first appeared on SK hynix Newsroom.

In this second episode of the series written by professors of Daegu Gyeongbuk Institute of Science and Technology (DGIST), Professor Gain Kim of the Department of Electrical Engineering and Computer Science will explain the features of I/O interface circuits. This article will cover how the circuits work, the use of different circuit architectures in modern systems, their various applications, and their evolution over the years.

Why are I/O Interface Circuits Important for Data Transfer?

Figure 1. Diagram showing the communication between chips carried out by a wired transmitter (TX) and a receiver (RX)

These electrical signals are essentially voltage information holding the binary numbers ‘0’ and ‘1.’ Although this might seem like simple information is being transmitted, it requires significant processing steps to transmit these 0s and 1s.

Figure 2. An archetypal pulse signal (single-bit) that experiences attenuation and distortion after passing through a wired circuit

One of the reasons for this lies in the fact that there is an enormous amount of output binary data that a processor must handle per unit time to ensure high-speed communication between chips. However, the number of pads that act as the pathway between the inside and outside of a chip is limited. As mentioned in the beginning, this is comparable to there being a large number of vehicles on a city highway with a limited number of lanes. If the number of data transfer channels is restricted like this, the flow of data in each channel—or metal wires—must be as fast as possible to transfer data without disruption. However, increasing the speed of data flow in a limited space of a channel will make it difficult to distinguish between the signals 0 and 1 when the data arrives at the other chip’s receiver. This is due to the signal experiencing attenuation (weakening) and distortion as it travels through the wire and reaches the receiver as in Figure 2. In general, there is a greater amount of attenuation and a higher degree of distortion to the signal the longer the wire gets. Taking these defects into consideration, it is the role of a circuit called an equalizer to restore the signal to its original form as much as possible.

What is the Role of an Equalizer?

Figure 3. A signal impaired by a wired channel is restored by the receiver’s equalizer (EQ) and shows a clear distinction between its 0s and 1s

The equalizer is divided into a transmitter that sends the data out to the wire and a receiver that receives the data from the wire. As for the signal that is restored by the equalizer, it will have a clear distinction between 0s and 1s again as shown in Figure 3. However, when the transmitter sends a 0 and the receiver reads it as a 1 or vice-versa, this misreading of signals is called a “bit error.” So, the equalizer reduces the occurrence of these bit errors—or the bit error rate. Transmitters and receivers that hold these functions are collectively referred to as a serial link, a wireline transceiver, or a SerDes (serializer-deserializer¹). It is the combination of these serial links that then form an I/O interface.

¹Serializer-deserializer (SerDes): A transmission system that sends signals over a high-speed connection from a transceiver on one chip to a receiver on another. The transceiver converts parallel data into a serial stream of data that is re-translated into a parallel form on the receiving end.

How Have I/O Interfaces Evolved?

Figure 4. A comparison of the design of high-performance processors (a) in the past featuring a single die (b) and a more recent design with multiple small-sized dies

In recent years, high-performance processors have increasingly been designed in the form of MCM. The first major reason for this is that when manufacturers previously attempted to improve the performance of processors, the die size was increased which led to a reduction in yield and consequently affected the product’s marketability. Thus, the design paradigm for high-performance processors has shifted from increasing the number of cores and the SRAM capacity within a single die to making a module containing multiple small-sized dies to act as a single processor. This can be seen in Figure 4.

Figure 5. Example of using a single-chip design to form a scalable accelerator package. Very low latency, low power, and high broadband interface performance are required between dies.

The second reason for the transition to the MCM design has to do with scalability. Figure 5 shows an example of forming multiple product lines of accelerators based on a single-chip design. Depending on the target performance level in parallel computation, the system can be configured with different numbers of dies in a single package. This, in turn, saves design costs and reduces the risks by eliminating the need to implement various chip designs.

An essential part of ensuring the scalability in performance is the I/O interface. If the interface encounters a bottleneck, performance improvements cannot be guaranteed when configuring multi-chip systems even if high-quality individual dies are used. As these MCM interfaces have very short wires and only suffer minimal signal losses, it is less necessary to utilize high-performance equalizers. However, since multiple dies must be naturally connected and operate as if they were a single die, the latency from transmitting and receiving data over the interface must be very low. In addition, the bit error rate needs to be close to zero even without the use of an error correction code (ECC)². Recently, progress has been made in reducing MCM’s latency by using a minimal amount of equalizers in die-to-die interfaces and focusing on increasing the total amount of data that can be transferred per second per the unit length of a die’s edge (Gb/s/mm).

²Error Correction Code (ECC): Algorithms that detect and correct data transmission errors to reduce the bit error rate.

Figure 6. Simplified diagram of a high-speed SerDes whose receiver includes CTLE, VGA, and DFE

The peripheral component interconnect express (PCle)³ SerDes, which is used to expand the connectivity of processors, is an example of another interface structure used for high-performance processors. The difference between the PCIe SerDes and the multi-die design shown in Figure 4 (b) from the perspective of the interface is the greater signal attenuation. In a PCIe SerDes, the signal loss in the wired channel—including package bumps, PCB wires, and connectors—can be up to 40 dB. So, focus is placed more on the equalizer’s ability for signal restoration rather than the latency in communication. Receivers that need to restore high losses reaching up to 40 dB utilize equalizers such as the continuous time linear equalizer (CTLE)⁴ and the variable gain amplifier (VGA)⁵, which are low-latency analog equalizers, in addition to the decision feedback equalizer (DFE) that has high-power consumption and a complex circuit design but good signal restoration capabilities.

³Peripheral Component Interconnect Express (PCle): High-speed, bidirectional serial data communication interface that interconnects devices.

⁴Continuous time linear equalizer (CTLE): An analog equalizer that compensates for channel distortions by boosting high-frequency components of the input signal.

⁵Variable gain amplifier (VGA): An electronic amplifier that varies its gain depending on the control codes.

As PCIe is highly versatile and generally the second-fastest interface that supports high-speed communication after the CEI/Ethernet SerDes for servers, vast research has been conducted recently on improving the performance of computing systems utilizing expansion cards based on the compute express link (CXL) protocol that operates over PCIe PHY⁶. Examples of commercialized products include the CXL-based DRAM expansion solution for servers. As for SK hynix, the company developed DDR5 DRAM-based CXL memory samples based on PCIe in August 2022. The CXL interface helps increase the efficiency of utilizing CPUs, GPUs, accelerators, and memory.

⁶PCIe physical layer (PCle PHY): A PCIe component that manages decoding, transmission, and reception of data for reliable PCIe serial communication between devices.

For ultra-high speed SerDes with lanes that reach more than 100Gb/s, it requires not only analog equalizers like CTLE and VGA but also digital equalizers such as digital signal processors (DSP) where a significant amount of equalization is conducted. Examples include CEI/Ethernet SerDes in data centers, and a number of fabless companies have recently released intellectual property cores that utilize DSPs in the 6th generation 64GT/s PCIe SerDes receivers. It is important to note that an essential block for the use of DSP is the ultra-fast analog-to-digital converter (ADC).

Unlike analog equalizers that operate in continuous time, DSPs run in gigahertz (GHz). Thus, an ADC operating at dozens of giga samples per second (GS/s) needs to parallelize its sampled output data. As the parallelized data restores its signal on a block-by-block basis over multiple DSP clock cycles in the DSP, this results in higher latency compared to analog equalizers. Even as of early 2023, DSPs that process parallel data from dozens of high-speed ADCs consume more power than analog equalizers with equivalent performance levels. However, signal losses that cannot be restored by only using analog equalizers can be restored by using a combination of analog and digital equalizers. So, receivers based on ADC and DSP are becoming more commonly used for ultra-high-speed, long-distance interfaces that can equalize channels with a maximum loss of 40 dB.

The Digitalization of the I/O Interface in the Future

We have briefly covered the basic concepts and roles of high-speed I/O interfaces and looked at the differences in the detailed structures of transmitters and receivers that are used for different fields. For applications where decreasing the latency of data transmission and reception is more critical than compensating for signal losses in channels, it is sufficient to use receivers that only utilize relatively low-speed analog equalizers. However, for ultra-high-speed interfaces that have high signal losses in channels, receivers with structures utilizing a DSP to get additional compensation for signals—on top of analog equalizers—are being used.

Design techniques have evolved over the past five years to “digitize” the various blocks of an ultra-high-speed wired transceiver. This trend is expected to continue for the foreseeable future as developments in the miniaturization of the semiconductor process provides more direct benefits to digital circuits than to analog circuits. Additionally, due to the expansion of the scope of DSP’s utilization, the structure of the equalizer—which has remained relatively simple in terms of algorithms—is expected to utilize more complex and diverse communication algorithms.

<Other articles from this series>

[DGIST Series] How the Quest for AI Led to Next-Generation Memory & Computing Processors

[DGIST Series] The Role of Semiconductor Technologies in Future Robotics

[DGIST Series] The Technologies Handling the Growing Data Demands in Healthcare

[DGIST Series] Silicon Photonics: Revolutionizing Data Transfers by Unleashing the Power of Light

[DGIST Series] AI-Powered Micro/Nanorobots to Revolutionize Medical Field

[DGIST Series] Sensor Interfaces and ADC Circuits: Bridging the Physical and Digital Worlds

The post [DGIST Series] How Broadband Interface Circuits Are Evolving for Optimal Data Transfer first appeared on SK hynix Newsroom.

Neurometabolomics, brain engineering, microlasers, biophysics, and semiconductor convergence technology are some of the research areas of Daegu Gyeongbuk Institute of Science and Technology (DGIST) in South Korea. In particular, the institute has heavily invested in semiconductor R&D, with its campus home to a fabrication facility.

With such vast knowledge and experience of semiconductors, the DGIST professors are the ideal authors of our new seven-part series which covers subjects such as the basic modules of semiconductors and semiconductor application cases, as well as other advanced technologies such as memory and interface circuits.

In the first episode of the series, Professor Jong-Hyeok Yoon from the Department of Electrical Engineering and Computer Science will explain how computing processes have evolved to power current AI technologies. As AI requires an enormous amount of computation power along with high speed and efficiency, this article will reveal how development in this field started with CPUs and GPUs and has progressed to digital accelerators that have transformed the function of memory.

The Evolution of AI

AI has been in the spotlight recently following the release of the AI chatbot ChatGPT which has captured the public’s imagination due to its ability to answer sophisticated questions. However, 20 years ago, AI was generally only of practical use on large-scale servers to perform tasks such as online searches using natural language. In comparison, AI on edge devices¹ such as personal computers was still at a very low level. People who remember the Office Assistant dog called Rocky that appeared in Microsoft Office programs more than two decades ago will probably agree. The dog was generally seen as an inconvenience to users as it was only able to provide information from its programmed index and would sometimes respond with irrelevant information. In addition, it was difficult to turn it off.

The progress made in AI technology over the years can be seen by simply opening smartphone camera apps. Nowadays, smartphones feature AI chips with an NPU² —which are imitations of a neural network—so that related photos can be pulled up by merely entering a search term, while it is also possible to detect and edit objects in photos even without the use of a server.

So, why were we unable to develop such advanced AI technologies in the past? The 2016 Go match between legendary South Korean player Lee Sedol and Google Deepmind’s AI AlphaGo program which attracted over 200 million global viewers was clearly not the first time people thought about the application of AI. The fundamental principles of AI were actually proposed in the 1940s, while its practicality was demonstrated from the 1970s to the early 2000s by renowned computer scientist Geoffrey Hinton and his research group who developed the Restricted Boltzmann Machine (RBM)³ and the backpropagation⁴ algorithm. Although the theory of AI has been established for a long time, its application has only recently begun to develop due to the limitations in computational capabilities for implementing AI and in the hardware resources required to save data such as weights⁵ and results of neural networks.

¹Edge device: Devices that are distinguished from existing smart devices in that their data processing takes place at the edge of the network rather than in the center of the network—or the cloud.
²Neural Processing Unit (NPU): A processor optimized to drive machine learning. Unlike GPUs that require the software to create and train artificial neural networks, an NPU is characterized by its implementation of artificial neural networks on a unit of a hardware chip.
³Restricted Boltzmann Machine (RBM): A generative, stochastic artificial neural network that can learn a probability distribution over its set of inputs.
⁴Backpropagation algorithm: A statistical technique used in machine learning for multilayer neural networks that calculates the error—the difference between the predicted value and the actual value—to accurately adjust the weight.
⁵ Weight: The amount of importance given to each individual value when calculating an average.

AI Hardware That Provides Optimal Digital Computation: CPUs, GPUs, and ASICs

Focusing on computational capabilities, neural networks used to implement AI require a large-scale matrix-vector multiplication (MVM)⁶ . Therefore, the purpose of AI hardware lies in how efficiently and quickly it can compute these large-scale MVMs. As a computer’s main brain, a central processing unit (CPU) can run a wide range of programs, but it is unable to support fast and effective MVM due to its limited parallel computation capability. To put it into perspective, if a CPU that is capable of processing 10 different tasks is required to perform 100 simple MVM computations, it would require the CPU to repeat its computations 10 times.

Given the increasing demand for parallel computing power, the graphics processing unit (GPU) was considered a turning point in AI development. GPUs have a large number of parallel computing⁷ units implemented to process graphic data required for multimedia tasks such as gaming and videos. Through general-purpose computing on GPU (GPGPU), in which the GPU performs computation for tasks typically handled by the CPU, it became possible to use AI in practice. However, despite the fact that a GPU can perform computations quickly as it can carry out immense parallel computations, it cannot operate efficiently. As previously mentioned, GPUs have many parallel computing units for graphical data processing, but their inefficiency is caused by their lack of dedicated MVM computation units (let’s remember that GPGPU stands for general-purpose computing on GPU). Moreover, as GPUs are intended for large-scale parallel processing operations, they consume a vast amount of power and therefore cannot support AI applications that require ultra-low power such as edge AI⁸ .

Thus, experts in academia and the industry have been involved in developing computing accelerators based on the ASIC⁹ design to provide fast and energy-efficient computation. The earliest ASIC chips for AI are primarily digital accelerators. It comprises many computing units specialized for large-scale MVM, and it also provides scalability depending on the AI network structure.

⁶Matrix-Vector Multiplication (MVM): An operation between a matrix and a vector that produces a new vector.
⁷Parallel computing: A type of computation in which vast quantities of information are processed simultaneously by breaking up the tasks and processing them all at once. It is the opposite of the serial processing method where a task needs to be completed before moving onto the next task.
⁸Edge AI: A method of AI computation that happens at the edge of a given network, usually on the device where the data is created instead of in a centralized cloud computing facility or offsite data center.
⁹Application Specific Integrated Circuit (ASIC): In contrast to a general integrated circuit, it is a non-memory semiconductor chip designed for a specific product.

So, can we say that digital accelerators are the ultimate solution to securing hardware resources for AI computation? Even if the efficiency and speed of computing units have increased with ASIC-based digital accelerators, we cannot tell if the efficiency of the entire system is directly improved. To calculate this, it is necessary to understand how much energy the entire operation takes.

In the Von Neumann architecture¹⁰, which is the most common computer structure, the computational devices read and process data from the memory and send it back to the memory. Thus, the efficiency of MVM operation in the Von Neumann architecture is affected by two factors: the energy required to deliver the inputs and weights to the computing unit, and the energy dissipated in multiplication. As inputs are directly fed to the computing unit, energy consumption is negligible. However, in the case of weights, it takes about 500 times more energy than the computational energy to transfer data from the external DRAM to the computing unit. During a DARPA workshop in 2020, Professor Philip Wong at Stanford University stated that energy consumption from memory limits the entire computational efficiency. Although there have been various efforts to increase this efficiency such as reducing the computational energy, most of the energy, in fact, was being used to read and write the weight in the memory. Consequently, to improve the computational efficiency of the entire system, the number of reading and writing operations should be reduced.

¹⁰Von Neumann architecture: A program-embedded computer structure typically featuring three levels consisting of the main memory unit, a central processing unit, and an input/output unit. Most computers today follow this basic structure, but its bottleneck limits the ability to design high-speed computers.

▲Figure 1. Energy consumption from computation and memory access (left) change from the Von Neumann architecture to the PIM computational structure (center and right)

IMC and NMC: Two Structures of PIM-based Accelerators for Computational Efficiency

The computing accelerator based on Processing In-Memory (PIM)¹¹ was developed from observing these occurrences while breaking away from the Von Neumann architecture. As PIM is a memory with computing capability, the input is fed directly to the memory where the weight exists. This structure allows the memory to compute on its own and output the result value. PIM can be generally categorized into In-Memory Computing (IMC)¹² and Near-Memory Computing (NMC)¹³ . The difference lies in whether one sees the PIM as a computation in a memory circuit or in a memory module. IMC modifies the memory cell itself to allow it to perform computations as an ASIC, while NMC refers to the integration of high-density memory such as HBM and ASIC specialized to compute MVM within a memory module—semiconductor substrates including memory chips. Note that in academia and the industry, the term “PIM” generally refers to IMC and NMC, respectively.

As the NMC structure still needs to read weights from the DRAM, one might assume that there would be a disadvantage in computational efficiency of the entire system. In the Von Neumann structure, the connection between the CPU and the memory is composed of multiple PCBs¹⁴ including the mainboard, memory module, and connectors. However, in contrast, NMC connects the memory with the computational ASIC within a single package through System-in-Package (SiP)¹⁵ or 3D IC technology, greatly reducing the energy and delay time caused from reading and writing memory. IMC goes a step further than NMC as it dramatically reduces the energy consumption and delay time by performing operations within the memory.

▲Figure 2. Examples of In-Memory Computing (left) and Near-Memory Computing (right)

¹¹Processing In-Memory (PIM): A next-generation technology that provides a solution for data congestion issues for AI and big data by adding computational functions to semiconductor memory.
¹²In-Memory Computing (IMC): A technique of running computer calculations entirely in computer memory. It enables extremely fast processing that helps businesses increase performance and analyze massive volumes of data in real time at very high speeds.
¹³Near-Memory Computing (NMC): Near-memory incorporates memory and logic in an advanced IC package, while in-memory brings the processing tasks near or inside the memory.
¹⁴Printed Circuit Board (PCB): A semiconductor board that’s made up of electronic circuits and has components soldered on its surface. These boards are found in most electronic devices.
¹⁵System-in-Package (SiP): A type of package that combines multiple devices into a single package to implement a system.

So, why does NMC still exist if IMC seems to be more efficient? Several of the reasons include variability in computation size, computation and memory density, and bandwidth. NMC can easily compose a PIM structure by still utilizing semiconductor memories, a field in which South Korea excels, while adjacently placing and adding computational ASICs which can support various MVM sizes. IMC, on the other hand, requires modifications to existing memory circuits to facilitate computation, resulting in a trade-off in density for high computational efficiency. This also leads to loss of storage capacity for weights and decreased throughput.

The Importance of Memory Capacity and the Development of Accelerators

In addition to computational efficiency, memory capacity for weights is also one of the important performance metrics in PIM. ChatGPT, the AI chatbot developed by OpenAI which is based on the GPT-3.5 language model, is made up of more than 175 billion weights. As each weight uses 16-bit floating point (FP16), it requires about 350 GB of storage for the weights alone. Since it is not possible to compute with 350 GB of weights loaded at once, the NMC’s computational ASIC or IMC circuit must be able to utilize many weights to reduce the number of weight updates. This results in a higher percentage of total operations being computational and less energy being spent on data transfers. Taking this into consideration, an NMC-based PIM system utilizing a highly-integrated HBM seems to be the more viable approach.

Besides large-scale AI systems, how does this relate to edge AI? There are many applications for edge AI where all the weights can be put on a single chip. As edge AI is often battery-powered and requires ultra-low power operation, the energy consumption from the data transfer between memory and computing units is prohibitive. Therefore, it is necessary for the edge devices to implement edge AI with all weights preloaded on computationally energy-efficient circuits such as IMC. For this reason, in addition to the computational efficiency of IMC-based PIM systems, the amount of pre-loaded weight capacity plays an important role in advancing edge AI.

In line with the industry’s research and development of NMC-based PIM systems, academic circles strive for advanced AI. Research is being conducted on the design of volatile memory-based PIM accelerators with SRAM¹⁶ , eDRAM¹⁷ , and DRAM, as well as next-generation, non-volatile memory-based PIM accelerators with RRAM¹⁸ , PCRAM¹⁹ , and MRAM²⁰ . Among volatile memories, SRAM has been actively researched due to the accessibility of the CMOS process. While there are methods such as current-based operations and resistance ratio that are being used, charge sharing and capacitive coupling methods that utilize the low process deviation of capacitors²¹ are the main research streams for SRAM-based PIM accelerators.

¹⁶Static Random-Access Memory (SRAM): A type of random-access memory (RAM) that retains data bits in its memory as long as power is being supplied. The term “static” differentiates SRAM from DRAM; SRAM will hold its data permanently while connected to a power source, while data in DRAM decays in seconds and thus must be periodically refreshed.
¹⁷Embedded DRAM (eDRAM): A dynamic random-access memory (DRAM) integrated on the same die or multi-chip module (MCM) of an ASIC or microprocessor.
¹⁸Resistive Random-Access Memory (RRAM): A type of non-volatile random-access memory (RAM) that works by changing the resistance across a dielectric solid-state material.
¹⁹Phase-Change RAM (PCRAM): A type of semiconductor memory which utilizes the phase changes of certain materials to store data. PCM possesses qualities of both flash memories and DRAMs. Like flash memories, PCM is non-volatile, meaning that it does not lose information even when the power is cut. Like DRAMs, PCM processes data quickly and is power efficient.
²⁰Magnetoresistive Random-Access Memory (MRAM): A type of non-volatile semiconductor memory which utilizes magnetic reluctance to store data. Like flash memories, MRAM does not lose information even when the power is cut, and like DRAMs it processes data quickly and is power efficient.
²¹Capacitor: A device that stores data in a semiconductor memory. It can be thought of as a data storage room.

▲Figure 3. Types of operations in SRAM-based PIM accelerators

Non-volatile memories have a higher density and computational efficiency than volatile memories and they are more suitable for ultra-low power edge devices as non-volatile memories do not require a continuous power source to maintain weights. Among non-volatile memories, MRAM has a low on/off ratio—the ratio of resistance values when expressing 1 and 0—which is disadvantageous for multi-bit encoding. As a result, PIM accelerators utilizing RRAM or PCRAM which have high on/off ratios, have instead been widely studied. Non-volatile memories still require additional peripheral circuits such as write-verify due to their immaturity that includes the large variation in resistance values by device. For this reason, non-volatile PIM accelerators are still inferior to volatile memory-based PIM accelerators such as SRAM in terms of density per total area and computational efficiency. However, as there is a lot of potential for advancement in the technology of devices, many R&D projects focusing on this subject are underway in South Korea.

Making AI More Practical

In the past, AI development has been hampered by the gap existing between theorizing the concept and a lack of hardware to bring it to life. Over time, the development of digital accelerators such as CPUs, GPUs, and Google’s Tensor Processing Units (TPUs) has led to dramatic improvements in the amount of computation that is possible. Furthermore, following the arrival of PIM accelerators, memory has expanded its basic role of storing data to making computations, essentially playing the role of the computer’s brain. Despite these advancements, there is still a long way to go regarding research on PIM accelerators compared to markets of digital accelerators like TPUs and NPUs. There also needs to be better understanding of characteristics in the circuit that include computational resolution, storage capacity, latency, and power consumption. To overcome these challenges, researchers should keep on conducting studies on improving the performance of hardware that can better support AI algorithms, as well as optimizing algorithms of AI neural networks that can better suit the characteristics of PIM accelerators. This implies that the combination of circuits and algorithms will become an important pillar of future PIM accelerator research.

<Other articles from this series>

[DGIST Series] How Broadband Interface Circuits Are Evolving for Optimal Data Transfer

[DGIST Series] The Role of Semiconductor Technologies in Future Robotics

[DGIST Series] The Technologies Handling the Growing Data Demands in Healthcare

[DGIST Series] Silicon Photonics: Revolutionizing Data Transfers by Unleashing the Power of Light

[DGIST Series] AI-Powered Micro/Nanorobots to Revolutionize Medical Field

[DGIST Series] Sensor Interfaces and ADC Circuits: Bridging the Physical and Digital Worlds

The post [DGIST Series] How the Quest for AI Led to Next-Generation Memory & Computing Processors first appeared on SK hynix Newsroom.