Google’s Auto ML Can Replicate Next-Gen ML Software More Capable Than Human-Made Systems

Google’s AutoML system, first announced in May 2017, has demonstrated the ability to replicate and produce other machine learning systems that are more powerful and efficient than the best human-designed machine learning systems that currently exist.

Google has been expanding and building upon its machine learning capabilities, with AutoML as just a part of it’s overall machine learning eco-system. The company recognized that the effort of developing machine learning systems from the ground up was resource intensive; Google scientists and developers spend a significant amount of time and effort establishing these systems, in which an exponential number of possible learning models could be explored. Further, there are few individuals who have the expertise and experience to do this on a large scale.

With AutoML, the goal was to automate the process of developing custom machine learning systems, which not only frees up the time of machine learning experts, but allows them to focus on refining and improving systems. 

Google focused on two algorithms for developing machine learning models: evolutionary algorithms and reinforcement learning algorithms. For AutoML, the reinforcement learning algorithm was the main method used.

In reinforcement learning, a task is identified to a controller which then produces a "child" system, which is then tested. The system evaluates the performance of the test and provides that feedback to the controller which then uses that information to propose another child system. This loop continues for thousands of iterations until the system is able to predict and learn which systems perform best.

Reinforcement learning diagram. Image courtesy of Google Blog.

Google used two benchmark datasets to test AutoML, involving image recognition and language modelling. The systems that AutoML produced were just as capable as, if not better, than the systems machine-learning experts produced.

The process has also given some insight into how some systems work, and why they may work better than others. For example, a multiplicative-combination in a recurrent network was identified in a produced network that human-machine learning experts would not have considered beneficial. But the AutoML-produced network identified that this was, in fact, useful for handling gradient vanishing, and discovered a new architecture. 

New architectures were discovered through reinforcement learning. Image courtesy of Google Blog.

AutoML, once completed, will make machine learning more accessible. Google’s intent is to eventually democratize AutoML so even more non-expert users can access machine learning. The experts can then devote their time to improving the next generation of the technology. Finally, for companies like Google, it will enable more trivial implementation of machine learning and allow for greater volumes of deployment.

Feature image courtesy of Google.