As a student, when I used to learn and implement machine learning models, I thought deployment was the final step for a Data Scientist. But my perception completely changed when I started my journey as a professional. I soon realised that in most of the practical data science use cases, the first deployment of a ML model means the job is only half done. There are several challenges that we face even after deployment.
They can be classified into two main categories:
1) Machine Learning or Statistics related
2) Software engineering issues
ML Related:
· Data Drift: X changes
Data drift is the change in the distribution of data over time.
For example: How Online Behaviour has changed from Q1 2020 to Q1 2021. If we have developed some model for budget allocation for a marketing team with these inputs (as shown in fig), we should probably retrain our model as we can see that Movie or TV streaming service is getting more and more popular with time.

· Concept Drift: X->Y changes
Concept drifts occur when the patterns the model learned no longer hold.
For example: If we have deployed a model to predict churn, after some time we can observe that the importance of different features changes with time. Hence the model needs to be re-trained.

We can see that the loading time is getting more importance for churning with time.
Software engineering issues:
· Environmental Changes: Some of the libraries used may run out of support. It’s better to keep an eye on them and update them as and when required
· Out of service Cloud: Suppose we have deployed some ML model on cloud and the cloud went out of service for some time. In that case, we can have a hybrid model deployed on the browser which can run offline too. One can check this link for a better understanding.
· Compute resources (CPU/GPU/memory): It might happen that we have trained our model using very powerful machines but that is becoming too expensive for the project.
· Security and Privacy: These are the areas where we can’t compromise. There are many best practices related to it. Like – Not directly using ID and password in code etc. You can refer to the link if you want to deep dive into it.
End Notes:
I hope this article will help you in foreseeing the challenges if you are looking for or have just deployed your model in production. The data science world is growing with each passing day. I would love to know if you have faced any other post-deployment challenges. Please write it down in the comment section.
References: –