News

# Leap Motion Raises $50 Million to Develop Gesture Tracking System July 24, 2017 by Robin Mitchell ## Leap Motion, a hand tracking company, has just raised$50 million funding for its continued development of its hand tracking system.

### The Leap Motion Platform

The Leap Motion Mobile VR platform is a combination of hardware and software to provide hand tracking so that the user can interact with a computer-generated environment with their hands as opposed to devices such as keyboards and mice. For comparison, the Oculus Rift may display an environment with perspective, but it's lacking in this ability to interact.

##### The Leap Motion VR headset. Image courtesy of Leap Motion.

The hardware is based around an ARM processor (CYUSB3014-BZXI), two small IR cameras, and several IR emitters. It can be mounted on most headsets to stream image data to a computer. The headset's dual infrared cameras, being wide angle lenses, obtain a nearly 180-degree field of vision.

Once the IR image has been captured, the ARM processor (a fast USB controller at 200MHz) stores this data in RAM and alters the data to optimize the resolution. Once this is done, the controller streams the data to a computer where Leap Motion Service decodes the images to produce the augmented reality data (i.e., the position of the user's hands and fingers).

Interestingly, the Leap Motion Software transmits the final VR/AR data over TCP, which allows for most internet-enabled devices to interact with the controller, including smart phones.

### Other VR/AR Users

Leap Motion is producing products for use with augmented and virtual reality, but who are they selling their software to? What end applications currently exist for gesture control and how do they hold up against classic forms of interfacing?

As it turns out, there are plenty of companies who are selling or developing products with gesture control in mind. One example is the DJI Spark drone which is entirely controlled with gestures as opposed to a controller that one would expect to find on an areal vehicle. The drone, launched from the palm, automatically flies away from the user and can be controlled with hand gestures. For example, the user can raise their hand to instruct the drone to fly upward and then make a square with their fingers to have the drone capture a high-tech "selfie" picture of the user.

However, while this may sound impressive, the gesture control has still yet to be improved as there are some concerns about the drone's responsiveness. But, if improved, such devices could prove critical in situations where a controller is impractical (such as disaster relief and military scenarios).

##### The DJI Spark, a drone controlled via gestures. Screenshot courtesy of DJI Tutorials.

Gesture controls are also finding their way into aid devices, including a glove that can be worn by those who are deaf. A hearing-impaired person may struggle to interact with interfaces such as Siri which rely on spoken commands. The gloves, still in research, can track hand movements and translate sign language into commands that can be understood by voice / text activated systems.

NASA's also been looking into gesture control and virtual reality for the purpose of training future astronauts. The move comes from the impracticalities of controllers used during VR training and so NASA aims to replace such controllers with TENZR straps.

### The Importance of Gesture Control in VR/AR

Virtual reality and augmented reality are starting to become commercialized—by the Oculus Rift, for example. But their uses go far beyond the applications of entertainment and gaming. Virtual realities allow for users to immerse themselves in simulated environments, a concept that could be applied in many extraordinary circumstances.

For example, students could use such technology to view the world from different timelines and immerse themselves in an environment and live history instead of reading about it. Archaeologists could use VR to view a historical site as it stood and gain insights to how ancient civilizations would have viewed such monuments.

##### VR and AR would greatly benefit from gesture control. Image courtesy of BagoGames [CC-BY 2.0]

For augmented reality to become viable, however, it must be integrated with advanced gesture control. Augmented reality relies on creating an environment that is superimposed on the current surroundings (such as Google Glass), and so interaction through gesture control is a no-brainier. Augmented reality could benefit many industries just like virtual reality where they can change and shape the world around them.

Implementing gesture control allows for true interaction with the virtual environment which opens up endless possibilities for many industries. For example, engineering firms could use a virtual environment to simulate their product while users can interact with the device as if it were real to show potential problems and improvements. The medical field could greatly benefit from such technology by training surgeons in an environment where surgeons could improve their motor skills and gain insight that could only normally be found in real operations.

The applications are nearly endless and they could have serious repercussions on how we see the world and how we learn.