AMD Claims First x86 Processor with Dedicated AI Hardware
Aiming to enable efficient AI computing in the mobile space, AMD at CES rolled out its Ryzen AI engine, along with processors that embed the new AI engine.
We’re back again with more CES 2023 news. Amidst a slew of processor rollouts at the show from AMD last evening, the company unveiled what it claims as the first x86 processor with dedicated artificial intelligence hardware.
In this article, we’ll examine the rationale behind dedicated AI hardware on a processor, delve into the details of the AMD’s Ryzen AI engine, and share insights from the press briefing with AMD’s Don Waligroski, senior product manager and Nick Ni, senior director for data center AI and compute markets.
“AI is the defining megatrend in technology,” said AMD Chair and CEO Dr. Lisa Su in last night's keynote speech at CES.
Last night’s CES 2023 show opened with the keynote by AMD Chair and CEO Dr. Lisa Su. A key statement by her was that “AI is the defining megatrend in technology [today].” Following through on that idea, a number of the company’s key announcements last night revolved around AI.
An Adaptive AI Architecture
In a briefing to the press, AMD’s Nick Ni introduced a new architecture from the company called XDNA. XDNA is AMD’s new adaptive AI architecture and Ni says the company is integrating it across its product roadmap. Explaining XDNA at a high level, NI says it’s really optimized for AI inference.
“Neural networks function like neurons where data flows from layer to layer,” says Ni. In this simple example (see below), there are multiple layers of neurons where each “dot” does some kind of compute function—like matrix multiply or convolution—and then you pass the data to the next neuron to be processed.
The XDNA architecture passes large arrays of AI data from layer to layer.
“There's quite a bit of sequential dependency from layer to layer,” says Ni, “To follow this, our AMD AI engine architecture works as an adaptive dataflow architecture where you can actually use a large array of compute, but then you can pass the data efficiently from array to array. This can be done without going out to external memory or even to some cache, which often consumes more power and results in a longer latency.”
“With this kind of architecture, we can achieve much higher performance, much lower latency overall, and very high energy efficiency. And it's also customizable for any particular AI workload the customer wants to do.”
Ni says that another key benefit of the XDNA architecture is that it’s extremely scalable. “We call this cloud-to-client symmetry,” saysNi. “We can shrink the array to something very small to fit into something like a laptop or even a tablet. And we can scale it for something much bigger with hundreds or even thousands of arrays.” Such huge arrays can be designed in data center and cloud infrastructure systems.
Along just those lines, among AMD’s AI-themed product previews last night was its Alveo V70 AI Accelerator board. Designed for AI and cloud infrastructure systems, the board can handle multiple AI inference workloads, and is available for pre-order now.
First x86 Processor with Dedicated AI Engine
Probably the most innovative among AMD’s many product announcements at CES last night was its new Ryzen 7040 Series Mobile processors that are the first to embed a dedicated AI hardware. Called AMD Ryzen AI, this hardware engine uses the XDNA technology discussed earlier.
AMD’s Don Waligroski explains that this engine is designed specifically to accelerate AI-oriented workloads, and do so balancing optimal performance and optimal power usage. Waligroski uses the analogy of GPUs to put this in perspective.
AMD Ryzen AI is what the company claims is the first dedicated AI engine on an x86 processor.
“At one time, graphics processing was done on the CPU,” he says. “CPUs are a great general purpose processor. But at some point, people realized that the graphics workloads just weren't served as optimally as they could be if there was an optimal piece of hardware designed and applied just to that problem.”
“That's the way we should be looking at this. This is a dedicated block of AI functionality. It sits on the processor in the same way that built-in integrated graphics sit on the processor.”
According to Waligroski, Ryzen AI can handle four concurrent dedicated AI streams. That means we can do AI multitasking,” he says. “We can run separate AI tasks at the same time. That's pretty unique because we have four streams, but we can also apply those to the same problem.
All About Efficiency in Processing and Power
Waligroski says it’s all about efficiency. “Because it's designed for AI and AI only, it’s also designed to be very efficient at that task,” he says. “And we're seeing efficiencies much higher than the competition.” The AI engine also offloads the CPU and graphics engines on a processor to do other things.
Importantly, the AI IP is optimized for long battery life. “You can run AI workloads with a minimal hit on your battery because it's just designed with efficiency as a target,” says Waligroski. “It really is designed to exceed the power efficiency of that of regular processors that you would normally run these kinds of workloads on. It's a data flow architecture that’s very robust and programmable.”
The first laptops with AMD’s Ryzen 7040 series processors and Ryzen AI are expected to ship in March.
This AMD Ryzen AI engine is integrated into several of AMD’s new Ryzen 7040 Series Mobile processors that were announced last night at CES. The 7040, including the HS versions shown in the image above, offer what AMD claims is the “fastest PC processor graphics in the world” with up to eight “Zen 4” cores and AMD RDNA 3 graphics. The AMD Ryzen 7045HX Series version of the series embeds up to 16 Zen 4 cores and 32 threads. The 7045HX processors are built on 5 nm process technology.
Dedicated AI on Our x86 Laptops?
According to AMD, the first laptops with Ryzen AI technology are expected to begin shipping in March this year. This move by AMD to bring AI hardware into the mobile realm is a sign that AI is becoming ever more pervasive in today's computing.
We’ve seen AI innovations on specialized computers, at the IoT edge, and it’s far from new as a software-based technology. Dedicated AI hardware moving onto PCs perhaps validates the message that AI is indeed the defining megatrend in technology today.
All images provided by and used courtesy of AMD