The world’s leading publication for data science, AI, and ML professionals.

From Models to Deployment: Learn MLOps for Free from Google and Stanford Experts

Machine Learning Operations (MLOps) is an emerging field and I strongly encourage you to learn more to catapult your data science career.

EDUCATION

Photo by Carl Jorgensen on Unsplash
Photo by Carl Jorgensen on Unsplash

Don’t Let Your Skillset Get Extinct

The data science field is evolving at an unprecedented pace. While the field is certainly not becoming extinct in the foreseeable future, your skillset may well do if you cease learning and upskilling.

Data science continues to enjoy the spotlight as more organizations wish to use data to stay competitive. This is promising for each one of us. However, the rising demand also means that an ever-increasing number of people are getting into data science.

With ubiquitous learning opportunities at everyone’s disposal, you must continue to learn and grow to stay competitive in such an environment.

Why is MLOps Important?

This post will introduce you to one of the nascent sub-disciplines within machine learning, MLOps (Machine Learning Operations).

Unless you work in a very large organization with a mature data science team, you will likely need a good working knowledge of MLOps. Even if you are keen on staying as a data scientist (and not a data engineer as the currently defined distinction), you better be open to the idea of learning about the MLOps side of things to work effectively as part of a "data science" team and to be able to eventually lead such a team.

No organization is really interested in investing money for you to develop models that stay in Jupyter notebooks. They are, instead, interested in solving a real-world challenge and wish to use models that can give them a competitive edge. There is no getting away from model production and deployment.

Intended Audience

If you have had little or no experience in deploying machine learning models, then you should definitely read on. However, even if you have hands-on experience with designing, developing, and deploying an end-to-end machine learning project, you may still find the perspective and links to relevant resources useful.


MLOps: A Bird’s Eye View

Let’s get something out of the way first. A common misconception is that the deployment of an Artificial Intelligence (AI) product or service is primarily about the machine learning (ML) algorithm. Once the ML algorithm is finalized, it could be deployed with standard software engineering practices. The reality is far more complicated. In fact, unlike traditional software requiring routine maintenance challenges, ML products/services have additional ML-specific issues that may exist at the "design" level rather than at the code level [1].

Figure 1 (adapted from [1]) shows the requirements of an ML system in production with the "ML Code" highlighted to give you an idea of the various challenges that need addressing once you decide to deploy your ML model.

Figure 1: The surrounding infrastructure to support the ML system is significantly more vast and complex compared to the ML code (adapted from [1] by the Author)
Figure 1: The surrounding infrastructure to support the ML system is significantly more vast and complex compared to the ML code (adapted from [1] by the Author)

In a real-world ML system, the ML code is a small part of a vast and complex infrastructure. This is a graphical illustration to help raise awareness and illustrate what ML model deployment could mean in practice, over and above what you may have learned in a typical machine learning course. If you are curious, you can read the original, highly cited paper elaborating on the various issues highlighted in the figure above [1].

The Present: Ad hoc Model Deployment

Attempting to deploy models is not new. Over the years, many Data Science practitioners have been dealing with a variety of challenges when attempting to deploy models.

During my Ph.D. days (almost a decade ago now), I had spent significant time developing algorithms that could automatically get breathing rate from a small, finger probe [2]. In a follow-on project, I had to implement this algorithm using data from COPD patients collected remotely [3]. At the time, this algorithm required running on a server (for access by researchers and clinicians). In that project, I dealt with several issues related to data flow (how and where the data is to be stored), latency and throughput requirements (overnight batch processing was sufficient in this particular application), an appropriate API between the algorithm block and the rest of the infrastructure, etc.

Breathing Rate Algorithm running on the server, overnight batch processing (Image by Author)
Breathing Rate Algorithm running on the server, overnight batch processing (Image by Author)

In a different project, I had to implement an algorithm that automatically processed brain signals and decoded them [4]. The concept here was to instantly decode brain waves from patients and then provide them feedback on screen. Unlike in the COPD case, this project required instant processing and this was, therefore, implemented locally in the same computer that was used to acquire signals.

Decoding brain signals to estimate force, decoded in close to real-time (Image by Author)
Decoding brain signals to estimate force, decoded in close to real-time (Image by Author)

In both projects, the requirement was to take an existing Machine Learning algorithm and then deploy it for actual use. However, the context was widely different and the issues faced were consequently different and required different kinds of software engineering challenges.

My experience when attempting to deploy models is likely the story of many experienced data science professionals. Back then, I had not heard of "MLOps". Most of us, at the time, were concurrently dealing with software-related issues while attempting to deploy an ML algorithm in an ad hoc manner.

The Future: MLOps

Typical machine learning courses delve into model building and the associated statistical concepts. Until recently, this made sense too since many organizations were still in exploratory phases having acquired data but not sure what needs to be done with it. It probably made sense to recruit a "data scientist" who acquires the data, analyzes it, and creates cool charts, and derives insights.

However, moving forward, this is not going to be sufficient in many cases. Organizations would need to deploy systems. At the end of the day, there needs to be an actual product or service.

The time is now ripe to bring all the model-deployment-related issues together in a systematic manner and teach them to the new breed of data science professionals.

This field, now gaining traction, is called Machine Learning Operations or MLOps in short. It is an emerging discipline concerned with the tools and techniques to support the progression of ML models from conception to deployment.

A Conceptual Framework for MLOps

An excellent framework to think about Mlops is to think through the complete machine learning project lifecycle (introduced by Andrew Ng in his MLOps introductory course) who stated:

When I’m building a machine learning system, I’ve found that thinking through the Machine Learning project lifecycle is an effective way for me to plan out all the steps that I need to work on.

Adapted from the course, Introduction to Machine Learning in Production by Andrew Ng (Image by Author)
Adapted from the course, Introduction to Machine Learning in Production by Andrew Ng (Image by Author)

In light of the framework above, MLOps can be defined as a data science discipline that consists of various tools and associated principles to support an ML project’s progress through its lifecycle.


Where Can I Learn From?

As this is an emerging field, I expect more resources to be available in the near future. You can find dedicated courses to help ML engineers supporting one of the three main cloud infrastructure providers (Amazon, Google, and Microsoft). You can also find relevant courses on Udacity, and Coursera.

However, the resource that I highly recommend is the one available on Coursera (linked from DeepLearning.AI).

It’s a specialization entitled "Machine Learning Engineering for Production (MLOps). You can take the courses for free. The courses in the specialization are:

1: Introduction to Machine Learning in Production

2: Machine Learning Data Lifecycle in Production

3: Machine Learning Modeling Pipelines in Production

4: Deploying Machine Learning Models in Production (this course is not yet released)


Concluding Remarks

The courses linked above are advanced. If you are a complete beginner, then I recommend starting from Andrew Ng’s course on Machine Learning, followed by the Deep Learning specialization first.

However, if you already have a working knowledge of Machine Learning, and intermediate Python skills including experience with any deep learning framework (TensorFlow, Keras, or PyTorch), then you are good to go.

This is an amazing opportunity to learn more about MLOps from some of the best in the world: Robert Crowe (TensorFlow Developer Engineer at Google), Laurence Moroney (Lead AI advocate at Google), and Andrew Ng (founder of Coursera, and now an adjunct professor at Stanford University).

Read every story from Ahmar Shah, PhD (Oxford) (and thousands of other writers on Medium)


References

[1] Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.F. and Dennison, D., 2015. Hidden technical debt in machine learning systems. Advances in neural information processing systems, 28, pp.2503–2511.

[2] Shah, S.A., Fleming, S., Thompson, M. and Tarassenko, L., 2015. Respiratory rate estimation during triage of children in hospitals. Journal of medical engineering & technology, 39(8), pp.514–524.

[3] Shah, S.A., Velardo, C., Farmer, A. and Tarassenko, L., 2017. Exacerbations in chronic obstructive pulmonary disease: identification and prediction using a digital health system. Journal of medical Internet research, 19(3), p.e69.

[4] Shah, S.A., Tan, H., Tinkhauser, G. and Brown, P., 2018. Towards real-time, continuous decoding of gripping force from deep brain local field potentials. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26(7), pp.1460–1468.


Related Articles