Overview of Your Journey
- Introduction – No Production, No Problems!
- Notebooks can be Used for Everything!
- Why Automate When You have the Time?
- Testing? Just Never Make Mistakes!
- Dependency Management in my Head!
- Wrapping Up
1 – Introduction – No Production, No Problems!
As data scientists, data engineers, and ML engineers we are bombarded with information about ML models in production. Hundreds of videos and thousands of blogs have tried their best to help us avoid the situation we find ourselves in. But to no avail. Right now, it pains me to say, there are ML models in production all around the world. Generating value to millions of unknowing people. On every street corner. And we tolerate it, just because it’s common.
When going to conferences, I listen to nervous data scientists on stage in front of big audiences talk about production. It’s clear from their forehead sweat and clammy hands that the situation is serious. This has been going on for many years, but we didn’t trust their prophesies. And look at us now. We should have listened.
This is no time for anyone to say "I told you so". We need to band together to retake what is ours. We need to to unilaterally profess disdain for the modern ways of doing things. We need to go back to better times, when ML models in production was just a nonbinding bullet point on job adverts for middle-sized companies.
Someone has to take the lead and guide this journey of redemption. And who better than me? Without intending to brag, I’ve made several ML models that did not reach production. Some of them were not even close. I can share with you some of the best tips so that you don’t need to replicate your development environment – because there will be nothing else.
I’ve clearly divided each following section into two parts. The first is called The Righteous Way. It tells you how to avoid ML models in production. The second is called The Sinful Way. This lets you know what to avoid, as this is a fast track to getting ML model in production. Don’t get them confused!
2 – A Single Notebook can be Used for Everything!
The Righteous Way
I’ve found that one of the simplest ways to avoid getting ML models in production is simply to have your entire code base in a single Jupyter notebook file. This includes model training, model predictions, data processing, and even configuration files.
Think about it. When all the sensitive configuration files are within your main document, it becomes virtually impossible to upload anything to GitHub or Azure DevOps without introducing major security risks. In addition, have you every tried to read a pull request from someone who has modified a file with 100.000 lines? The best response is to simply drop the remote hosting and version control all together.
This, my friend, puts us on the fast track to avoiding production. No remote hosting means that silos will be inevitable. And have you stopped to think about the cloud consumption costs of running the entire model training every time you need a prediction – remember that we only have a single Jupyter notebook for both training and testing. The single notebook architecture as I’ve proudly named it is simply genius.
The Sinful Way
Instead of the single notebook architecture, people have over time gone astray. They’ve started to split their code, configurations, and documentation into multiple files. These files unfortunately follow conventions that makes them easy to navigate. There are even tools like Cookiecutter that provides easy-to-use templates structuring ML projects.
The code itself is often divided into modular components like functions, classes, and files. This encourages reuse and improves the organization. Coding standards are enforced with PEP8 standards and automatic formatting tools like Black. There are even perverse blog posts guiding data scientists to better software development practices. Disgusting!
Not only is the source code structured. There are tools like Data Version Control (DVC) for versioning big data, or tools like MLflow for structuring ML model packaging. And none of these are even particularly difficult to get started with. Stay clear of this!
3 – Why Automate When You have the Time?
The Righteous Way
Back in the day, things were different. We felt ownership with our code. This ownership came from revisiting the code routinely with regular time intervals. That’s right, we ran all the code manually. This is clearly the way god intended, and I am tired of pretending that it’s not.
The manually approach gave you complete control. There was no need to check if a data pipeline had been initiated at 3 a.m. – you were there! Meticulously typing out execution commands from memory every night. When you have the time, why rely on CRON expressions or Airflow DAGs to get the job done? At the end of the day, can you really trust anyone?
This manual approach turned out to be a blessing against ML models in production. Data drift happens to ML models, and then the models need to be retrained. But with the manual approach, someone have to sit put and watch for when sufficient data drift has happened. In this economy? Not likely! It’s probably better to drop the whole production aspect and return to better ways.
The Sinful Way
Let’s talk about the elephant in the room: Continuous integration & Continuous delivery (CI/CD). I know, it makes my blood boil as well. Nowadays, people automatically check for code quality and run tests before updating the code base. Tools like GitHub Actions or Azure DevOps Pipelines are also used to automate retraining of ML models in production. Do we even stand a chance?
There is more! People now use tools like Terraform to set up infrastructure necessary to support ML model in production. With this infrastructure as code approach, environments can be replicated across a wide range of settings. Suddenly ML models in production in Azure are suddenly in production in AWS!
There are also tools like Airflow that help with the orchestration of data pipelines. Airflow ensures that preprocessing steps wait for the previous steps to complete before executing. It also gives you a GUI where you can inspect previous pipeline runs to see how things have gone. While these things seem simple, they can catch errors quickly before they propagate through your system and corrupt your data quality. High quality data is unfortunately key for successful ML models in production.
4 – Testing? Just Never Make Mistakes!
The Righteous Way
The philosophy of testing is a response to countless production failures and security breaches. The idea is to catch as many issues as possible before launching a product. While indeed noble, I think this is misguided. A much more simple approach reveals itself for those who pay attention. Precisely! It’s clear as day once you see it. You should simply not write any mistakes.
With mistake-free code, there is no need for tests. Even better, while you are 100% certain about the validity of your code, others are less so inclined. They will doubt the correctness of your code. As such, they will avoid putting your ML models in production in the fear that everything will break. You know it wouldn’t. But they don’t have to know that. Keep you pristine coding skills to yourself.
Not writing tests have the additional advantage that you get less documentation. Sometimes tests are vital for letting others understand what your code should be doing. Without tests, even less people will bother you, and you can continue on in peace.
The Sinful Way
Writing tests has become common, even for ML models and data engineering pipelines. Two major classes are unit tests and integration tests. Unit tests, as the name suggests, test a single unit like a Python function whether it works as intended. Integration tests on the other hand test whether different components work seamlessly together. Let’s focus on unit tests.
In Python, you have the built-in library unittest for writing, you guessed it, unit tests. This is OOP-based and requires a bit of boilerplate code to get started. More and more people are using the external library pytest to write unit tests in Python. It is functional rather than OOP-based, and requires less boilerplate code.
Writing unit tests also has a side effect. It forces you to write code modularly. Modular code that is well-tested break less often in production. This is bad news for your ML models. If they don’t break in production, then they will stay there forever. When you think about it, then ML models breaking in production is them trying to escape. And who could blame them?
5 – Dependency Management in my Head!
The Righteous Way
Managing dependencies is an important part of any ML project. This includes for instance knowing which Python libraries you’ve used. My advice is to simply remember which libraries you’ve installed, the operating system you are using, the runtime version, and so on. It’s not that hard, I’m sure there is even an app for keeping track of this.
I do sometimes wake up at night and wonder if I have version 0.2.3 or version 0.3.2 of scikit-learn running. It’s no matter! All the versions exist, so there shouldn’t be a problem…right? If I didn’t routinely solve dependency conflicts, then my dependency conflict skills would go stale.
The advantage of simply remembering your dependencies is that it becomes difficult for others to run your code. Especially if you don’t want to tell them all the dependency details. In this way, you can avoid someone suddenly getting the sneaky idea to lift your ML models to production.
The Sinful Way
People with bad memory skills have opted for the easy way out. They’ve looked for solutions that handle your dependencies for you. I swear, people get lazier every day! A simple approach to managing runtime versions and library dependencies is to use a virtual environment and a requirements.txt file. In Python, there are tools like virtualenv that allows you to set up virtual environments easily.
Those that want to go a step further use a container-based technology like Docker. This can replicate everything from the operating system and upwards. In this way, sharing ML models becomes pretty effortless as long as everyone knows how to run Docker containers. In modern tools like Azure Machine Learning, there are easy ways to use standardized Docker images. These can include commonly used libraries like scikit-learn and tensorflow. And you don’t even have to remember the versions yourself!
6 – Wrapping Up
I hopefully don’t have to point out that you DO want Machine Learning models in production. The points I’ve argued for here can be reversed and help you get those pesky ML models in production successfully.
Feel free to follow me on LinkedIn and say hi, I’m not nearly as grumpy as I pretended in this blog post ✋
Like my writing? Check out some of my other posts for more Python content: