Does it matter that there’s more than one AutoML?
We see Machine Learning (ML) as the application of software to try to fit models and predict the future based on historical data. It can be visualised as a person behind a control panel with a bunch of knobs and levers, tuning them until they get to the best combination that delivers the most accurate outcome. The data scientist’s job is to tune the control panel with all their knowledge of all the different algorithms and how they work.
‘Automated ML,’ is a way to automate the construction of a Machine Learning model, helping the data scientist avoid spending a lot of time on unnecessary grunt work which is commonly performed to get models up and running. It’s a compelling idea: a tool that speeds up your process, compresses the time spent writing code and automatically checks assumptions done manually. This will allow you to focus on two essential tasks. You can focus more on utilizing your data science expertise and combining it with your understanding of the business to fine-tune your high-value machine learning models. This allows you to focus and manage model building on exception. It also allows more time to interpret the results and deliver them to stakeholders.
We used to talk about Third Generation Languages versus Fourth, where the latter was supposed to automatically propagate the complete business program you wanted, instead of you doing the hand-coding and remembering why your instructor at Code Camp said global variables were always a bad idea. On beginning a career in Machine Learning, one might spend a lot of time downloading different tools that have just one algorithm. You would download a tool just for one problem and another tool for another, and you would put them all together and have a mash-up of components. If you couldn’t find an implementation, you’d have to implement something yourself. Nowadays, we have libraries that make it easier to try a whole array of algorithms with the same interface. Time’s being saved, and productivity increased for the data science team.
Different approaches are needed because we have different problems
It would be naive to think that by replacing ML with AutoML, all our problems will be solved. It’s not that simple, as there are a whole variety of great AutoML systems out there, Open Source and proprietary, and that’s as it should be. The reason for that is that there’s no one-size-fits-all algorithm or even a combination of algorithms. No one tool is going to solve all your problems. It appears that those 4GLs back in the 1980s and 1990s didn’t take all the programming load away. The same is true here, as you can’t just press a button and have it done for you. You can have a friendly environment set up to make it easy for you, but you still need to do the work.
Almost every AutoML tool is unique because there’s no one way to solve a problem. Of course, at the abstract level, every tool is ‘the same’ in that you put data in, then you train a model, and it can make predictions, but they need to have a different internal architecture. Different approaches are necessary because we have various problems in various industries, so image recognition isn’t the same as text understanding, for example. Given this, some implementations of these algorithms work better, and some work worse.
You shouldn’t just choose any AutoML because it’s free or it’s Open Source, as it has to match the problem. The first step is to decide what type of question you’re addressing, and then, within that, you’ll be limited to a specific subset of the tools. If you’re doing image classification, you have some tools available which are usually very different from the ones someone’s spun up for tabular data.
Look for diversity in the algorithms
It’s also important to know about your own data that needs to be ingested; is it big, is it small, is it wide, as there are all kinds of shapes? If your data is something in an Excel spreadsheet then you can go one way if it’s images, another, if it’s text, this is the best route of travel.
That should orient you quickly and efficiently and cut down the search space for the AutoML you want to use. The next step is to use the AutoML with your data to see if it works. This is essential because almost every vendor will tell you that they are good at everything because they want you to buy the subscription.
Try a test patch up-front. First, look for some amount of diversity in the algorithms that are used. (Hint: you want a range of traditional statistical approaches, tree capability and Deep Learning, as there’s not just one algorithm that’s always going to win.) Tools that can do some kind of ensemble of algorithms are usually a good approach. It is also not just about the algorithm. Additional manipulation of crucial attributes (a method known as feature engineering) will always deliver an uplift in accuracy, so make sure there is a range of approaches taken. The challenge is that the combination of engineered features considered increase exponentially, so it is vital to make sure that your AutoML can quickly get to the optimal model.
Ultimately, what we need to remember about multiple AutoMLs is that by conducting work upfront, evaluating lots of different and diverse types of models, will help address problems down the line. At this point, organisations can either just make a selection from that group or try to combine them into an ensemble. Any excellent AutoML tool should do that, and if it’s only looking at one type of problem or one kind of technique, then it’s not what you need. You need what’s best to help you deliver the best AI that your question deserves.