Data Centers Are Overloaded. The Inventor of FPGAs Is Swooping In With a “Comprehensive” SmartNIC
The FPGA-based SmartNIC arena—including giants like Microsoft and Intel—just got more interesting. Enter the inventor of FPGAs.
Datacenter operations are facing a crisis. While compute cycles per server are increasing gradually, network port speeds are increasing exponentially.
Xilinx asserts that "port speeds are outstripping Moore's law." Image used courtesy of Xilinx
The solution may lie in SmartNIC platforms, especially FPGA-based SmartNIC platforms. One such platform is based on Xilinx's newly-released Alveo U25.
Xilinx's level up in the SmartNIC arena is noteworthy for two reasons.
For one, Xilinx is going up against tech giants like Microsoft with its Azure SmartNIC and Intel with its FPGA programmable acceleration card N3000.
The second (and more striking) reason is that Xilinx literally calls itself "the inventor of the FPGA." Considering that Xilinx's new SmartNIC—along with Microsoft and Intel's SmartNICs—are FPGA-based, does Xilinx have an edge on its big-name competitors?
What Is a SmartNIC?
Before delving into the business politics of Xilinx's latest SmartNIC device, it may be helpful to lay the groundwork of what we mean by "SmartNICs."
NIC stands for a network interface card, according to Kevin Deierling from Mellanox Technologies (another SmartNIC manufacturer). The backbone of a NIC is a PCIe that attaches to a server and enables an interface into an Ethernet network.
The Alveo U25 is the first SmartNIC to kick off Xilinx's new platform. Image used courtesy of Xilinx
In the most basic terms, a SmartNIC's job is to offload operations that network system CPUs would otherwise have to tackle. These might include TCP/IP acceleration, HTTP processing, establishing firewalls, and handling time-consuming encryption duties.
SmartNICs are a boon to throughput for busy internet servers. They can also boost speed and efficiency in advanced security and machine learning applications as well as for telcos.
Three Types of SmartNICs
SmartNICs can be based on ASICs, systems on a chip (SoCs), or field-programmable gate arrays (FPGAs). Like the devices on which they are based, each has advantages and disadvantages.
ASIC devices, for example, are expensive to develop and inflexible, but unit costs are low. The SoC type typically includes an onboard CPU.
The Xilinx Alveo SmartNIC platform, along with its Microsoft and Intel counterparts, is based on FPGA technology.
Of the task types typically assigned to SmartNICs—compute, storage, and network—only FPGAs hit the mark on all three.
What SmartNICs Is Xilinx Up Against?
Microsoft also bases its Azure SmartNICs on FPGAs. The company’s stated goal is offloading host networking to hardware. Azure SmartNICs implement Microsoft's accelerator network (or "AccelNet") and are deployed on Azure servers deployed since 2015. Microsoft claims that AccelNet offers less than 15μs VM-VM TCP latencies and 32 Gbps throughput.
Microsoft Azure SmartNIC boards (Gen one on right, Gen two on left) and bump-in-the-wire architecture. Image used courtesy of Microsoft
Intel, in a whitepaper on building a PoC of segment routing using FPGA SmartNIC and P4 language, also advocates an FPGA solution and employment of the P4 language.
Architecture of Intel FPGA PAC N3000. Image used courtesy of Intel
Authors of the white paper reveal that Netcope P4, an FPGA-vendor that provides integration into different "flavors" of FPGA-based SmartNICs, offers up to two times the 100 GbE network capacity "to fully deliver on the improved efficiency over NPUs."
How Does Xilinx Fare?
As mentioned previously, most networks have not taken advantage of SmartNICs. Xilinx will face much competition in the race to fill the gap.
But Xilinx is not new to accelerator card technology. The company has previously offered the U50 and the U250 data center accelerator cards. In the past, we've discussed how the U50 was giving traditional processors a run for their money with its ability to fit into a PCIe slot, save power, and improve throughput and latency. We've also highlighted the U250 as the base of SK Telecom's AI inference accelerator, which is used in security systems to secure public and private spaces.
Xilinx distinguishes its latest SmartNIC platform as the only comprehensive option on the market, converging network, storage, and compute acceleration. The Alveo U25 kicks off a new line of Alveo SmartNICs, which are said to deliver a range of turn-key applications, including Open vSwitch and IPSEC. The SmartNIC platform, Xilinx says, is also fully programmable, using the company's Vitis unified development environment.
Xilinx's SmartNIC is described as the only comprehensive one of its kind. Image used courtesy of Xilinx
With its paramount FPGA authority, Xilinx asserts that the new SmartNIC platform includes a programmable FPGA to handle network flows. "Each flow can be individually delivered to the host and/or streamed in hardware to through bump-in-the-wire network acceleration functions and/or compute acceleration kernels for application processing within the FPGA," the company explains.
Stand-Out Features of the New SmartNIC
Xilinx's SmartNIC platform can take on some of the tasks now being shouldered at the network’s core, offloading cloud servers and freeing them up to attend to their own specific tasks. The end goal is continued seamless data flow. And yet, according to Xilinx, a full 80% of cloud server nodes have yet to avail themselves of this vital technology.
The goal of Xilinx's SmartNIC platform is to simplify and cut costs of data center infrastructure, provide more turn-key applications for compute problems, and "optimize or extend functionality leveraging programmability and IP plugins."
Diagram of Xilinx's SmartNIC performance stack. Image used courtesy of Xilinx
The Alveo U25 is based on Linux and includes two 10/25G ports and two PCIe Gen3x8. The FPGA itself features 6 GB DDR4 SDRAM, a quad Arm A53 processor complex, and over 520K LUTs.
The device also offers stateless and tunneling offloads. For manageability and pre-boot, Xilinx designed secure firmware updates into the Alveo U25. In addition, the accelerator card offers FPGA bump-in-the-wire acceleration, including machine learning, data analytics, and video transcoding.
Xilinx asserts that it has a leg-up on competitors with its "powerful FPGA, enabling hardware acceleration and offload to happen inline with maximum efficiency while avoiding unnecessary data movements and CPU processing."
The Xilinx U25 “onloads” tasks so the network kernel can free itself up by offloading them. This eliminates memory copies, context switching, lock contention, and high interrupt rates.
Onload technology is said to reduce latency by 80%. There is almost no jitter and TCP-based application performance is improved by as much as 400%.
Onload technology frees up the network’s kernel. Image used courtesy of Xilinx
Onload technology also bypasses the network kernel, freeing up its CPU cycles for more fundamental tasks. It establishes a direct connection between applications and the network. The technology makes fast connect and disconnect times possible, allowing for stronger connectivity.
Onload technology is designed to integrate easily into existing infrastructure and is compatible with industry-standard POSIX interface APIs. There are no software changes required.
Although Xilinx is a much smaller company than Microsoft or Intel, its authority in FPGAs may give the new SmartNIC a leg up—especially in terms of Xilinx's claim that it is the only comprehensive SmartNIC of its kind.
As SmartNIC platforms continue to address the networking challenges in data centers—the boom in network traffic, the demand on computing resources, and the limitations of network offloading—designers may want to keep an eye on how the founder of FPGAs is mapping out a solution with new accelerator cards.
Xilinx touts “its authority in FPGAs”. This is a bit hollow given that Intel’s authority in FPGAs comes from it’s acquisition of Altera. For decades, Xilinx and Altera continuously leap-frogged each other in the FPGA market for bragging rights concerning speed and logic size. The Microsoft-Intel/Altera collaboration seems pretty formidable.