News

Intel Announces the Company’s Largest Architectural Change in 40 Years

December 19, 2023 by Duane Benson

Intel's new Core Ultra mobile processors and 5th Gen Xeon processors build in AI acceleration with specialized cores, marking the biggest architectural change since the 80286.

At Intel's 2023 "AI Everywhere" event, Intel introduced a radical update to its processing architecture, represented in its mobile Core Ultra processors and desktop Core Ultra processors to be released in 2024. The architectures combine conventional high-capability CPU cores with special-purpose cores for low-power tasks, graphics acceleration, and AI acceleration. The newest 5th Gen Xeon CPUs, also announced at the event, focus on server performance and add in co-processor cores for cloud AI acceleration.

 

Core Ultra mobile processor

Breakaway view of the Core Ultra mobile processor

 

According to Intel’s company vision, the future of AI processing is in both the cloud and the edge. The company predicts that by 2028, 80% of all PCs will be “AI PCs”, equipped with AI co-processors.

 

Intel Moves to Neural Processing Units

Intel's AI co-processors, called neural processing units (NPUs), are its newest major innovation. When combined with the other special-purpose CPU cores, Intel believes that the new processors will increase overall performance while reducing electrical power draw and lowering overall total cost of ownership (TCO).

The diversified architecture of these devices combines multiple specialized cores assembled in a chiplet-based system. Chiplets enable higher yields by reducing the silicon real estate for each processor and optimizing silicon wafers for each chiplet’s purpose. Like most AI accelerators, Intel's NPUs rely heavily on multiply-accumulate (MAC) units. MAC speeds up complex multiplication operations by reducing the need to shift data between memory and registers.

 

Intel 5th Gen Xeon processors

Intel 5th Gen Xeon processors with AI acceleration built in

 

Intel released Xeon and mobile processors in late 2023, with desktop PC processors using Intel 4 scheduled for 2024. Intel's 4 process technology is a 7-nm geometry process, which Intel claims will increase clock speeds by 20% for the same power usage over Intel's 7 process (10 nm). 

The Core Ultra and 5th Gen Xeon are the first 7-nm processors from Intel and the first Core CPUs with smaller process geometries since 2019.

 

From Numerous Identical Cores to Targeted Specialized Cores

Conventional cloud AI processing uses graphics processing units (GPUs) and tensor processing units (TPUs) for massive parallel processing and matrix math optimization. For this reason, mainstream CPUs from Intel have long included GPUs. With Intel's previous architecture, the main CPU cores were tasked with everything regardless of the compute load. This would result in low-load tasks using more electrical power than necessary and taking CPU cycles away from high-load processes. It would also leave specialized math-intensive processing to the main CPU cores, which are not optimized for special-purpose operations. 

Intel’s Xeon server processor and their Core mobile and desktop CPUs have historically relied on speed and optimized software as a workaround. The new system, with multiple specialized cores, is a radical departure from the “one size fits all” philosophy of just increasing the number of identical cores.

 

Tackling Cloud AI Head On With Gaudi3

In addition to the new NPU co-processors, Intel announced a successor to the Gaudi2 deep learning AI accelerator. The Gaudi3 AI accelerator is targeted at cloud computing, large-scale deep learning, and generative AI systems. Intel claims that the Gaudi3 offers a 4x increase in BF16 performance over Gaudi2.

The BF16 (brain floating point) number format is used to improve floating point math performance in AI calculations. It is a 16-bit variation on the IEEE 754 float32 format. BF16 maintains the 8-exponent bits of standard float32, but only keeps 8 mantissa significant digit bits rather than the 24 bits in float32. AI benefits more from the speed increase garnered by using 16-bit math than it loses with the reduction in mantissa precision. Gaudi3 will also double the networking performance and deliver 1.5x the bandwidth of Gaudi2.

 

Hitting Cloud AI, Edge AI, and Large System Processing

With the Xeon server CPUs, workstation/laptop CPUs, and the Gaudi3 accelerator, Intel has broadened its AI to reach just about all of the key AI arenas. The Gaudi3 will find a home in massive AI systems. The 5th Gen Xeons will land in server farms to combine data handling and traditional server activities with accelerated AI capabilities. And, the Core Ultra mobile and desktop CPUs will bring AI to personal users.

 


 

All images used courtesy of Intel

3 Comments
  • P
    paulvsheridan December 23, 2023

    Great informative article, thank you.

    Like. Reply
  • estinamir December 23, 2023

    Computer evolution from CPU to GPU to NPU, chiplet time

    Like. Reply
  • jmdodd95682 December 26, 2023

    Oh please. Really? The biggest change some the 286??? I think the addition of multithreading and multicores was a bit bigger of a change. It changed everything in the x86 architecture. Or what about moving the memory controller into the CPU along with graphics???

    Pick your words more carefully.

    Like. Reply