“Cobot” with Deep Learning and Gesture Recognition Hits Audi Production Floor

March 29, 2017 by Kate Smith

Audi has announced that a collaborative robot is working the floor in its industrial factory.

A collaborative robot, or "cobot", is joining the workforce on the manufacturing floor at Audi Brussels. "Walt", as it's called by its creators, can utilize deep learning to recognize and communicate coworkers via gestures—it can even learn new tasks by watching and imitating humans.

Is manufacturing about to experience the Fourth Industrial Revolution? Luc Can den hove is the president and CEO of imec, an R&D firm specializing in nanotech that largely orchestrated Walt's creation. According to him, Walt is the harbinger of what he terms the "Fourth Industrial Revolution, which is characterized by concepts such as automation, human-computer interaction, the Internet of Things and cloud computing".

Whether or not Walt is the beginning of a revolution for industrial robotics, it's certainly a victory for human-robot collaboration (HRC) and communication.

Project ClaXon

Walt, itself, is the culmination of many collaborations. On the academic side, Vrije Universiteit Brussel and the University of Hasselt. They worked with a total of industry partners, including Audi Brussels and Robovision, to develop Walt's interaction software for industrial applications.

imec.icon is a research program through Imec which aims to bring together researchers for two-year programs focusing on nanotech and digital tech. Project ClaXon was their initiative to gather researchers who could help develop better HRC for industrial settings, running through 2015 and 2016.



Walt is the fruit of Project ClaXon's labors. It's currently earning its stripes on the production floor of a factory at Audi Brussels in Belgium.

The Evolution of Industrial Robots

In the past, robotics in industrial settings have been somewhat quarantined, placed a safe distance from human workers. This is because they were programmed with set tasks that they weren't capable of deviating from in a responsive manner.

Removing the physical barriers between robots and human coworkers does result in some negative side effects. For example, the robots, themselves, must become more flexible in a physical sense. If a robotic arm should swing into a coworker, it needs to be able to give way so as not to harm the decidedly more delicate human. Adding springs to robotic arms helps with this issue, but also reduces the accuracy with which the arm can operate. This is why one of ClaXon's foci was to develop adaptive control software to make up for this inaccuracy. The new software they created increased accuracy for flexible robots by 60%.


Walt in action. Image courtesy of Audi Brussels


According to Patrick Danau, the General Director at Audi Brussels, the cobot on their factory floor uses a suite of sensors to "see" its environment and its coworkers. This includes sensors to detect depth, heat, and electrical current, as well as color cameras. These allow Walt to receive and interpret gestures.

Gesture Recognition and Responsiveness

Walt is designed to work with human coworkers seamlessly. This is why a second research focus was dedicated to enabling better communication between robots and humans. The team determined that an intuitive method of communication for cobots is through gestures. Walt is able to understand commands via gestures and communicate this understanding via a "face". 

This is directly tied to one of the most important abilities Walt boasts: imitation. Walt can watch a human demonstrate a task and then perform it. This is obviously hugely valuable when it comes to the time and costs associated with programming robots with tasks.

Robovision contributed much of the deep learning research that allows Walt to recognize individual coworkers and respond to their gestures.

A pivotal benefit of such collaborative robotic software is that it allows any coworker to interact with robots in the workplace, not just specialized engineers.

To learn more about research going into communications between robots and humans, check out Iorek, a robot developed by Brown University to better facilitate object retrieval.


Walt could very well be the first in a new wave of advanced industrial robotics. Hang on tight for that Fourth Industrial Revolution.