What is Population-Based Training and its Uses

Introduction

In this blog, we discussed What is Population-Based Training and its Uses. Population-Based Training (PBT) connects and expands sequential and parallel optimization techniques. A population of models and their hyperparameters are jointly optimized by Population Based Training (PBT), a straightforward asynchronous optimization approach, to maximize performance. Importantly, PBT finds a schedule of hyperparameter settings rather than attempting to identify a single fixed set to employ throughout the whole training process, which is typically a sub-optimal approach. This strategy enables robust and reliable model training with just a minor modification to a traditional distributed hyperparameter training framework. By optimizing over a number of hyperparameters, PBT is effective on deep reinforcement learning tasks, with faster wall-clock convergence and higher final agent performance.

What is Population-Based Training

Population-Based Training (PBT) is a combination of the two methods for hyperparameter optimization that are most frequently used: random search and hand-tuning. In a random search, a population of neural networks is independently trained in parallel, and the best-performing model is chosen at the end of training. This typically results in a small proportion of the population being trained with effective hyperparameters while a large proportion being taught with ineffective ones, wasting computer resources.

How Population-Based Training works

Similar to random search, PBT begins by simultaneously training several neural networks with random hyperparameters. Instead of training independently, the networks instead use data from the general population to fine-tune the hyperparameters and allocate computational resources to models that exhibit promise. This is modeled after genetic algorithms, where each population member, or “worker,” is able to use data from the rest of the population. A worker might, for instance, replicate the model parameters from a worker who performs better. The current values can be randomly changed to explore new hyperparameters.

This process of exploiting and exploring is carried out periodically as the training of the population of neural networks advances, ensuring that all the workers in the population have a good baseline level of performance and that new hyperparameters are consistently explored. As a result, PBT can instantly take advantage of effective hyperparameters, allocate more training time to effective models, and, most importantly, can adjust the hyperparameter values over the course of training, resulting in automatic learning of the most effective configurations.

Population-Based Training and its Uses

Generative Adversarial Networks (GAN), which are notoriously challenging to tune, can be successfully trained using PBT. The Inception Score, which is a gauge of visual fidelity, was specifically optimized using the PBT framework, which led to a notable improvement from 6.45 to 6.9.

One of Google’s cutting-edge machine translation neural networks, which are typically trained using meticulously hand-tuned hyperparameter schedules that take months to complete, was also subjected to it. With PBT, it was discovered that hyperparameter schedules in the same amount of time as it normally takes to complete a single training run that matches and even outperforms current performance. PBT is very beneficial for training new neural network architectures and algorithms that include new hyperparameters.

Also read – Differentiable ILP: learning explanatory rules