New Chip Purportedly Offers the “Best Memory of Any Chip for Edge AI”
USC researchers have announced a breakthrough in memristive technology that could shrink edge computing for AI to smartphone-sized devices.
As artificial intelligence continues to permeate our daily lives, computing demands on underlying hardware are becoming more stringent. Today, our most sophisticated and large-scale AI models exist in the cloud, like those behind ChatGPT. To fully unlock the potential of AI, many believe that it will be necessary to bring these models to the edge. Achieving this will require a combination of highly optimized, lightweight AI models as well as dense and powerful computing resources.
This week, researchers from USC made news in the industry with the publication of a new paper claiming to achieve “the best memory of any chip for edge AI”. In this article, we’ll talk about the need for improved memory for edge AI and the new memristive technology from USC.
The team at USC team believes their memory discovery could enable powerful AI algorithms to run even on small portable devices. Image courtesy of USC Viterbi School of Engineering
AI's Need for Better Memory
As AI algorithms become more complex, the need for faster and more efficient memory technologies becomes increasingly important. However, traditional memory technologies, such as dynamic random-access memory (DRAM) and Flash memory, are not suitable for AI applications due to their limited capacity, high power consumption, and slow read and write speeds.
A memristor is characterized by both charge and flux. Image courtesy of CMC
One promising technology for AI applications is memristors, which are resistive switching devices that can be used for non-volatile memory and neuromorphic computing. Memristors offer several advantages, including high density, low power consumption, and fast read and write speeds. Additionally, memristors can be used to implement neural networks, which are critical for many AI applications.
To bring AI to the edge, improved memristor technologies must support high-precision programming and ensure uniform and accurate performance across a massive number of devices. High-precision memristors are essential for downloading synaptic weights obtained from cloud training and programming them directly into memristors. This would make it practical to train neural network models from scratch and distribute them to billions of memristive neural networks at the edge.
Memristor Challenges
Memristive switching devices are known for their relatively large dynamic range of conductance, which can enable a large number of discrete conductance levels within the device. These unique conductance levels within a single device could be the key to achieving high-density memory for edge AI applications.
A common scheme of memristive neural networks for edge computing on a large scale. Image courtesy of Yang et al.
Despite the theoretical potential for an infinite number of conductance levels, the highest number of conductance levels reported to date has been no more than two hundred. A major reason for this is the device fluctuation that occurs at each conductance level that limits the achievable conductance range.
To guarantee uniform and accurate performance across a massive number of memristive networks, developers must have access to high-precision programming. This capability can enable many distinguishable conductance levels on each memristive device.
USC Unlocks 2,000 Memristor Conductance Levels
This week, researchers from USC published a new paper in which they describe a breakthrough in memristive technology for edge AI applications.
In the paper published in Nature, the researchers report the achievement of over 2,048 conductance levels for memristive devices, a number that represents the largest amount of discrete conductance levels among all types of memories ever reported. The researchers discovered that memristive fluctuation can be significantly reduced by applying specific electrical stimuli as a denoising process.
An electrical stimulus successfully denoises memristor conductance levels. Image courtesy of Yang et al.
Using this denoising process, which does not require any extra circuitry, the researchers successfully programmed a commercial, semiconductor manufacturer-made memristor into 2,048 conductance levels, corresponding to an 11-bit resolution. Importantly, this feat was achieved in memristor devices fully integrated into a chip in a commercial foundry—proving the prototype's commercial viability.
With this initial success, the researchers are hopeful they may have paved a path for massive AI models to operate on smaller computing platforms.
