Notes from Industry

Machine Learning Scaling Options for Every Team

Ryan Feather
Towards Data Science

--

Photo by Kamil Pietrzak on Unsplash

Machine learning has myriad applications, but integrating machine learning into a software application comes with substantial challenges! Machine learning(ML) algorithms are complex, can require large binary parameter files, and are computationally intensive. Additionally, there is often value in improving machine learning approaches as new data and technologies become available which means that your approach for incorporating ML in your application must be flexible.

Most importantly, the skills needed to build and train great machine learning algorithms are not the same skills needed to build and scale great software (though there is overlap). To get the best results, ML practitioners need the ability to train and iterate on algorithms with a wide variety of tools purpose built for that task. Software engineers aim to produce applications that are effortlessly deployed, scale effectively, and produce reliable results. These requirements mean that a team that applies machine learning at scale must optimize against several difficult objectives.

I’ve personally encountered many sides of this problem. When I was a software engineer working on an R&D team, I wanted to make sure that algorithms worked as well in production as they did in notebooks and scripts, and that integrating them with the product was cost effective. I thought a lot about concerns like automated testing, API definition, and scaling. As a data scientist, my focus was on training not just accurate but also generalizable models that would inform our users’ actions and that would be hard for our competitors to replicate. Now, I lead teams that are concerned with all of these goals and see how both sets of concerns can add to complexity that might change the return on investment of a feature from “game changing win” to “not worth it”.

Fortunately, there are several approaches to scalably integrate machine learning algorithms into applications. Deliberate technology choices can make it possible for teams with different skill mixes to build ML applications. This article will introduce a non-exhaustive list of approaches to scalably executing machine learning models.

First though, let’s consider an example service to put the upcoming solutions in…

--

--