Microsoft and Qualcomm Aim to Remedy Fragmented Computing Ecosystem
With a new development kit on the horizon, Microsoft and Qualcomm intend to help developers envision many different AI scenarios.
The computing hardware industry is more fragmented today than ever before. Today, there are hundreds of companies releasing new CPUs, GPUs, and accelerators to the market, often using different architectures and ISAs.
To the developer, this fragmentation has presented a significant challenge in code portability and versatility. To address these challenges, Microsoft announced this week that it has teamed up with Qualcomm to release, among other things, a new AI development kit.
Project Voltera is designed to simplify the complex NPU ecosystem for developers. Image courtesy of Microsoft
In this article, we’ll take a look at the rise of neural processing units (NPUs), the challenges of market fragmentation, and Microsoft and Qualcomm’s newest offering.
The Rise of NPUs
As new, complex computing tasks have arisen in recent years, engineers have faced the challenge of balancing power and performance in their computing infrastructure.
This conflict has led to the rise of the hardware accelerator and heterogeneous computing, in which specialized computing blocks tackle data and compute-intensive workloads. The result of this architecture is a system that gets the best of both worlds: high performance, real-time computation at lower power consumption than a general-purpose computing architecture.
Heterogeneous computing has become the norm for data and compute-intensive workloads. Image used courtesy of Cornell University
Of these hardware accelerators, one of the most prominent has been the NPU, which is designed to accelerate machine learning workloads. As a response to the high demand for these devices, the industry has inundated the market with a flood of NPU options.
Challenges for the Developer
Hundreds of companies globally produce their own NPU offerings. This trend is also size agnostic, ranging from the tech giants like NVIDIA and Apple to small-stage startups. This abundance of NPUs has presented a non-trivial challenge to the developers who write code for these chips.
Due to the sheer variety on the market, these offerings lack uniformity. Each NPU may use a different architecture and ISA, and hence require different considerations of the code that will run on it. This fragmentation has essentially forced developers to write different code for different ecosystems, environments, and chipsets, limiting scalability, interoperability, and progress as a whole.
Microsoft and Qualcomm Team Up
This week, Microsoft announced the release of a number of AI development tools at its annual Microsoft Build event. Among these tools, one of note is Project Volterra, which is the result of a collaboration between Microsoft and Qualcomm.
Microsoft is aiming to reduce the challenges caused by market fragmentation. Image used courtesy of Microsoft
According to the companies, Project Volterra is a development kit featuring AI capabilities and a comprehensive Arm-native developer toolchain. Project Volterra will eventually produce a hardware device powered by Qualcomm’s Snapdragon compute platform. While few details are currently known, Microsoft claims that the development kit will come with an NPU that has best-in-class computing. To support this collaboration, Qualcomm also released a new Neural Processing SDK for Windows.
Project Volterra will be accompanied by a new cross-platform development pattern meant to provide developers with a set of cloud-based tools to automate code cross-compatibility. The new platform will use ONNX Runtime and Azure Machine Learning along with an AI toolchain to enable developers to make their code compatible with any given NPU, regardless of the chipset or ecosystem.