Hailo Spins Vision SoC Crafted for AI Analytics-capable Smart Cameras

The idea of embedding sophisticated artificial intelligence (AI) into a camera has long been considered a costly and difficult engineering feat. Aiming to buck that trend, today Hailo has announced its Hailo-15 family of system-on-chips (SoCs).

The devices are designed to be integrated directly into intelligent cameras in order to enable advanced video processing and analytics at the edge.

 

In its FCBGA 15 mm × 15 mm packaging, the Hailo-15 can be easily embedded into a smart camera.

 

In this article, we analyze the capabilities and features of the Hailo-15, and we share perspectives from our interview with Orr Danon, CEO of Hailo.

 

Building on the Hailo 8’s Success

According to Danon, the purpose of the Hailo-15 SoC is to be able to run advanced video analytics by providing the ability to process multiple complex deep learning applications at full scale and at high efficiency.

The Hailo-15 builds on the company’s previous product the Hailo-8. Over the past couple years, the Hailo 8 has been integrated into a wide variety of end products, mostly video aggregators and gateways, says Danon. The Hailo-8 has also been embedded in many third-party board-level and box-level computing products.

While the Hailo 8 was positioned as a neural processing unit (NPU), the Hailo-15 was conceived as a vision processing unit (VPU). “Hailo 15 is what we call an AI VPU,” says Danon. “It’s a vision processing unit that is highly tailored to do extensive AI tasks.”

The Hailo-15 VPU family includes three variants: the Hailo-15H, Hailo-15M, and Hailo-15L, with performance levels of 20 TOPS (tera operation per second), 11 TOPS, and 7 TOPS respectively. The different versions enable smart camera designers to meet different processing needs and price points.

Importantly, the Hailo-15’s neural network (NN) core enables the processing of multiple advanced deep learning (DL) models in parallel. All three Hailo-15 VPUs support multiple input streams at 4K resolution and combine CPU and DSP subsystems with Hailo’s AI core.

“All three Hailo-15 cores are significantly more capable than anything in a camera form factor that I'm aware of in the market,” says Danon. “And coupling them with our existing software stack provides flexibility.”

Danon says this is a point that is very important to many of their customers that have deployments in cameras and gateways that want to rely on the same software stack. “We maintain the same software package that supports both the Hailo-8 and Hailo-15," he says. “So you invest once and can deploy into multiple camera form factors.”

 

A Pyramid of Computing Types

The Hailo-15 SoC architecture packs multiple computing engines into one device. The NN core enables high frame-per-second (FPS) deep learning for fast and accurate detection of more objects per frame. The DSP enables vision processing and high-quality video encoding. 

Meanwhile, an image processing unit (ISP) pipeline provides capabilities like low-light functionality, image stabilization, and image enhancement. And a quad-core Arm A53 with up to 1.3 GHz 12k DMIPs performance provides control and management.

 

Block diagram of the Hailo-15 shows multiple compute engines.

 

“The whole idea is heterogeneous computing,” says Danon. “Here we have like a pyramid of computing."

 

"First there is the Arm CPU—very flexible, very efficient, of course—but not that powerful. Next is the DSP/ISP to take care of the middleware tasks. And then there is the AI NPU to carry the heavy load of AI processing. We provide the full package of a full SoC that is required to drive a smart camera.”

 

Advanced AI Analytics at the Camera Edge

The significant capability of the Hailo-15 SoC is how it enables AI analytics to be processed right inside the camera. It can run several AI tasks in parallel, including faster detection at high resolution. According to Danon, this means you can identify smaller and more distant objects with higher accuracy, and fewer false alarms.

Providing examples, Danon says the Hailo-15H is capable of running the advanced object detection model YoloV5M6 with high input resolution (1280 × 1280) and at a real time sensor rate. It can also run the industry classification model benchmark, ResNet-50, at an impressive rate of 700 FPS. 

 

Example depicting AI-based smart camera technology used for smart city safety purposes.

 

High performance, local AI processing can enable some powerful use cases. As the company says, with Hailo-15, smart city operators can detect and respond to incidents faster. Manufacturers can boost productivity and machine uptime using sophisticated vision analytics. And In the transportation space, authorities can use the camera technology for everything from recognizing lost children to finding misplaced luggage to preventing accidents.

 

All images used courtesy of Hailo