How Fast Can a Neural Network Chip Recognize Images? TU Wien Says Mere Nanoseconds

March 18, 2020 by Robin Mitchell

Researchers at TU Wien are at last addressing the need for instantaneous image recognition.

A team of researchers from TU Wien has created an image recognition chip that can recognize images in tens of nanoseconds.


The chip researchers used (center) connected to a frame. 

The chip researchers used (center) connected to a frame. Image used courtesy of Joanna Symonowicz, TU Wien

How was this feat achieved and what applications will the device be used for?


A Brief Review of Neural Networks

Neural networks consist of weighted nodes that are interconnected to inputs and outputs. The system is fed example data and the output is compared to the true answer. The weighting of the nodes is then adjusted until the output matches the true data.

Then, other data is shown to the network and this learning process is repeated until it is able to reliably recognize data and produce the correct output.


Perceptron neural network

Perceptron neural network. Image used courtesy of Robert Keim

For a more in-depth discussion on neural networks, read into Robert Keim's fourteen-part series on neural networks, starting with a discussion on how to perform classification using a neural network.


The Need for Instantaneous Image Processing

Neural networks are the foundation of current AI technology for image recognition—up to 1,000 FPS on large powerful systems with more common systems handling up to 100 FPS.

While this is acceptable for non-critical applications, it can be a drawback in applications that call for fast image recognition. For example, researchers may need to develop a smart combustion system that can analyze the combustion process of an engine in real-time, allowing them to make a decision on fuel/air mixtures.

In such a situation, operators would benefit from a device that can process images almost instantaneously. This is what one team of researchers has claimed to have accomplished.


The Neural Image Sensor

The reason for the lag in neural nets is that such systems require multiple steps; the image is taken, passed to a neural net, and finally processed. Once processed, appropriate signals and other responses can be generated. But this whole process is reliant on a traditional CPU, which operates on discrete clock cycles.

To work around this obstacle, a team of researchers from TU Wein has created a neural image sensor that combines all steps into a single package that can recognize images in under 20 nanoseconds.

According to the study, which was published in Nature, their device was based on a "reconfigurable two-dimensional (2D) semiconductor, photodiode array, and the synaptic weights of the network are stored in a continuously tunable photoresponsivity matrix."


Imaging ANN photodiode array.

(a) an image of an ANN photodiode array; (b) diagram of one pixel in the photodiode array; (c and d) Schematics of the classifier. Image used courtesy of Lukas Mennel, et. al

The sensor the researchers used is similar to a traditional image sensor, which consists of an array of photodiodes that record images projected onto the chip. The photosensors are made of tungsten diselenide, an ultra-thin material that is only three atoms thick. These photosensors are connected to a number of output elements.

Researchers trained the chip by exposing the sensor to an image and using a computer program to adjust the sensitivity of each pixel. They did this by adjusting the local electric field around that sensor. The pixels on the sensor were then altered until the output of the chip matched the image being shown to the chip following any neuron action. 

Once trained, the chip no longer required the computer used to adjust the chip and the chip continued to recognize the images—despite not requiring a host computer. The output of the chip was ready within 50 nanoseconds.


Time-Critical Applications

The current sensor used by the TU Wien researchers only has 9 pixels, but it is already able to recognize different shapes. Because every pixel is connected to each weighted neuron, it can detect a wide variety of patterns.


A chip provides an appropriate output signal after analyzing an image

A chip provides an appropriate output signal after analyzing an image. Image used courtesy of Joanna Symonowicz, TU Wien

While this sensor would not be practical for recognizing complex images, it has real potential in high-speed environments, including fracture mechanics and particle detection. The implications of this research might also affect manufacturing environments where barcodes and printed numerical data is passed through a sensor at high speeds.

The fact that the sensor does not consume any electrical power during operation also means that the sensor may be highly applicable in low-energy environments, including disposable electronics and low-energy IoT systems. 



Have you heard of other strides in neural network technology that you'd like to share? Let us know in the comments below.