Everyone who’s done even a bit of machine learning has had to struggle with the intricacies of getting a model to work outside toy problems. There are literally hundreds if not thousands of knobs you can turn to make training more efficient, more robust, or even just to make it converge. Machine learning as a science has come very far in the last couple of decades, but as engineering, it is still in its infancy.
Enter AutoML or Automated Machine Learning. Sounds almost oxymoronic, or tautological: what does it even mean to automate machine learning?
is an emerging subfield of machine learning that attempts to progressively optimize most of the process for solving a typical machine learning problem. At a very high level, we can think of this process as a series of steps such as data collection and preparation, feature engineering, model selection and tuning, deployment, monitoring, and so on.
Each of these steps includes a lot of technical decisions with a myriad of options. Just take model selection: are we going full-in with neural networks, or should we try some of the basic models first? Which ones, for the matter? And don’t get me started with neural networks, we have to pick among layers, activation functions, regularizers, optimizers… If we look at data preparation, again, there are a plethora of options: how to impute missing values (or whether to do it at all), how to encode the data, which filters to apply… You get the point. With AutoML we can reduce the cognitive load of these million decisions significantly, though it’s no silver bullet, of course.
One of the most successful subtasks in AutoML is model selection and tuning. Say you want to try all the classifiers in scikit-learn
. You can loop through all the classes (fortunately they share a common API), but you also have to try different parameters in each class, whose valid value ranges are different for each class. And then each model has to be trained and cross-validated, how many times? Instead, you can use something like auto-sklearn
, a library that wraps all scikit-learn models under a single class AutoClassifier.
The world of AutoML is already huge and keeps growing. There are lots of open-source libraries you can use today, and the big players in the industry are already packing their cloud platforms with these tools.
Be aware, though, that the field is still in its infancy, and it’s no free lunch. As AutoML is basically machine learning on steroids, all the existing challenges are multiplied. The cost of training an AutoML model is orders of magnitude larger than training a single model, although a lot of research is being put into reducing it. And two major issues that remain ahead are related to the explainability and the intrinsic biases encoded in these models.
In the meantime, next time you have to train a somewhat conventional model on a somewhat conventional dataset, make yourself a favour and use the many AutoML libraries available. You’ll be surprised at the results and, even if you still need to tinker with details, you will at least start with a very good baseline to build upon.