Generative AI Startup Shows Off Digital In-Memory Computing Platform

Last month, Santa Clara-based startup D-Matrix announced Jayhawk II, the second generation of its AI compute platform purpose-built for accelerating big data workloads. As AI models become more prolific, developers require dedicated hardware to support the complex operations for inference. As such, hardware such as the Jayhawk platform could be invaluable to the future of data center computing.

 

By combining the latest advancements in VLSI, software, and packaging, the Jayhawk II platform can rapidly accelerate AI performance and keep operating costs low. Image used courtesy of Forbes

 

While basic machine learning models can run on standard computer hardware such as CPUs or GPUs, this hardware becomes impractical as models scale to data center sizes. In these instances, dedicated AI compute modules provide not only more performance but potentially a lower overall operating cost.

D-Matrix, a young company aiming to be a leader in AI-focused hardware, has set out to address this scaling by "transforming the economics of large multi-modal model inference in the data center."  

 

D-Matrix Targets Digital In-Memory Compute

Analog compute-in-memory is one of the up-and-coming techniques to tackle the scaling problem of large-scale AI deployments. D-Matrix has developed a similar approach with its Jayhawk and Nighthawk platforms.

 

To provide a robust chiplet interconnect, D-Matrix uses a "Bunch of Wires" approach to provide a low-latency, high-bandwidth chiplet-to-chiplet interconnect. Image used courtesy of STH

 

D-Matrix employs both digital in-memory compute (DIMC) to offload computation cost and a unique chiplet architecture and interconnect to develop a low-latency, high-bandwidth device. On bandwidth alone, the latest Jayhawk II platform can outperform high-end GPUs by up to 40x, translating directly to improved performance and better throughput.

It's these specs that have earned D-Matrix support from investors such as Playground, Microsoft’s M12, Nautilus, and many others.

 

A Scalable, "Lego Block" Architecture

In terms of raw performance, D-Matrix reports that the Jayhawk II platform offers up to 150 TOPs, floating-point numerics, compression for generative AI models, and up to 20x better inferences per second compared to high-end GPUs. This performance boost is due largely to the DIMC and chiplet architecture but can also be attributed to the scalable nature of the device.

 

D-Matrix's previous generation of AI hardware, Nighthawk, highlights the scalability afforded by chiplets, allowing designers to scale AI compute up or down as needed. Image used courtesy of D-Matrix

 

With the Jayhawk II platform, designers can scale based on the required performance, with options from 30 TOPs to 150 TOPs. This eliminates the common tradeoff between unnecessary performance or slower speed when choosing a GPU. Thanks to D-Matrix's “Lego block” approach, the modular Jayhawk II and Nighthawk platforms can be scaled to each designer’s needs. The Jayhawk II is currently available for demo and evaluation, and the publicly available Corsair chip is expected in late 2023.

 

D-Matrix Thinks Big With the Data Center

While developers can use GPUs, AI-specific processors, or edge-based systems as a short-term solution for AI challenges, the industry must eventually adopt a more wide-scale approach like D-Matrix's to upgrade data center-level AI performance.

Regardless of whether the Jayhawk II platform will be the victor in the AI hardware race, its availability could benefit not only designers looking for innovative ways to deploy their AI models but also companies seeking higher data center AI performance at a lower cost.