A Deep Dive on Winbond’s Memory Technology for Edge AI

AI computing ranks as one of the most important technology areas for semiconductor memory.

For AI edge computing, Winbond offers a memory technology called customized ultra-bandwidth elements (CUBE) that is designed to enable optimized high-performance running generative AI on hybrid edge/cloud applications.

In this article, we take a deep dive into this technology and examine the problems it solves for engineers developing edge AI computing systems.

High-Bandwidth Memory Needs of Generative AI

Generative AI—and AI in general—depend on the muscle of large models to digest intricate patterns and relationships within their respective domains. This drives up computing demons, which include both processing and memory.

In order to host AI applications, memory chips and modules must offer sufficient performance across several parameters. High bandwidth is only part of the puzzle— power efficiency, size, and thermal management are also critical in edge AI designs.

Bandwidth vs. density—the scalability advantage of CUBE for edge computing. (Click on the image to enlarge)
 

Existing memory solutions face limitations in memory bandwidth that impact performance. Physical characteristics, such as the number of IC pins, data-transfer rate, and the width of the memory bus, all play significant roles in interface bandwidth. 

Increasing memory bandwidth comes with lots of challenges. At higher speeds, signal integrity gets tricky as factors like attenuation, crosstalk, and reflections put hurdles on the path to increasing memory bandwidth. Meanwhile, increasing bandwidth can come with increased power consumption and even new thermal management challenges. These problems are particularly dire in compact, battery-powered edge devices.

How CUBE Addresses the Shortcomings of Conventional Memory ICs

To deal with all these challenges, Winbond’s CUBE technology employs these three aspects:

• Novel approaches to increasing I/O count and raising data speed
• Support for through-silicon via (TSV) technology as an option
• A 3D architecture that reduces thermal dissipation issues

CUBE enhances the performance of front-end 3D structures such as chip on wafer (CoW) and wafer on wafer (WoW), as well as back-end 2.5D/3D chips on Si-interposer-on-substrate and fan-out solutions.

CUBE's I/O interface supports a data rate of 2 Gbps with 1K I/O, providing total bandwidth ranging from 16 GB/s to 256 GB/s per die. In this way, CUBE ensures accelerated performance that exceeds industry standards and enhances power and signal integrity through micro bump or hybrid bonding.
 

CUBE 3D stacking

CUBE offers from 1 to 8 Gb/die based on the D20 specification. Providing flexible design and 3D stacking options, CUBE accommodates smaller form factors. The use of through-silicon vias (TSVs) further enhances performance, improving signal integrity, power integrity, and heat dissipation.

TSV Boosts Performance, Lowers Cost

By using TSV technology, along with micro bump/hybrid bonding, designers can shrink power consumption and save SoC design area. Efficient 3D stacking, leveraging TSVs, eases integration with advanced packaging technologies. By enabling reduced die size, CUBE lowers device costs and boosts energy efficiency thanks to shorter power paths.

The mix of increased bandwidth and energy efficiency, improved response times, and support for customization and compact die sizes makes CUBE well-suited to enable the full potential of edge AI as well as in hybrid edge/cloud use cases.

Winbond actively engages with partners, including IP design houses, foundries, and OSATs. System designers can gain access to CUBE technology through all those ways.

A Scalable Path for Edge AI Designs

Today’s demanding edge AI applications need a scalable roadmap forward to next-generation system designs. Technologies like Winbond’s CUBE memory technology represent a key tool for this path forward. To learn more, be sure to check out Winbond’s website. And, if you’re in Munich, Germany this week, please visit Winbond at its booth A5-153 at the Electronica 2024 trade fair.

All images used courtesy of Winbond