Why is it So Hard to Integrate Machine Learning into Real Business Applications?
You’ve played around with machine learning, learned about the mysteries of neural networks, almost won a Kaggle competition and now you feel ready to bring all this to real world impact. It’s time to build some real AI-based applications.
But time and again you face setbacks and you’re not alone. It takes time and effort to move from a decent machine learning model to the next level of incorporating it into a live business application. Why?
Having a trained machine learning model is just the starting point. There are many other considerations and components that needs to be built, tested and deployed for a functioning application.
In the following post I will present a real AI-based application (based on a real customer use case), explain the challenges and suggest ways to simplify development and deployment.
Use Case: Online Product Recommendations
Targeted product recommendations is one of the most common methods to increase revenue, computers make suggestions based on users’ historical preferences, product to product correlations and other factors like location (e.g. proximity to a store), weather and more.
Building such solutions requires analyzing historical transactions and creating a model. Then when applying it to production you’ll want to incorporate fresh data such as the last transactions the customer made and re-train the model for accurate results.
Machine learning models are rarely trained over raw data. Data preparation is required to form feature vectors which aggregate and combine various data sources into more meaningful datasets and identify a clear pattern. Once the data is prepared, we use one or more machine learning algorithms, conduct training and create models or new datasets which incorporate the learnings.
For recommendation engines, it is best to incorporate both deep learning (e.g. TensorFlow) to identify which products are bought “together”, and machine learning (e.g. XGboost) to identify the relations between users and products based on their historical behavior. The results from both models are then combined into a single model serving application.
The serving application accepts a user’s ID, brings additional context from feature and user tables, feeds it into a model and returns a set of product recommendations.
Note that serving must be done in real-time while the user is still browsing in the application, so its always better to cache data and models. On the other hand, recent product purchases or locations may have significant impact on future customer product choices and you need to constantly monitor activities and update feature tables and models.
An online business requires automation and a CI/CD process applied into machine learning operations, enabling continuous applications. It is important to support auto-scaling and meet demand fluctuations, sustain failures and provide data security, not to mention to take regulatory constraints into consideration.
The Machine Learning Operational Flow
In a typical development flow, developing code or models is just the first step. The biggest effort goes on making each element, including data collection, preparation, training, and serving production-ready, enabling them to run repeatedly with minimal user intervention.
The data science and engineering team is required to package the code, address scalability, tune for performance, instrument and automate. These tasks take months today. Serverless helps reduce effort significantly by automating many of the above steps, as explained in my previous post Serverless: Can is Simplify Data Science Projects?. Other important tools to keep in mind are Kubernetes and KubeFlow, which bring CI/CD and openness to the machine learning world. Read more about them in my post Kubernetes: The Open and Scalable Approach to ML Pipelines.
Machine Learning Code Portability and Reproducibility
A key challenge is that the same code may run in different environments, including notebooks for experimentation, IDEs (e.g. PyCharm) and containers for running on a cluster or as part of an automated ML workflow engine. In each environment you might have different configurations and use different parameters, inputs or output datasets. A lot of work is spent on moving and changing code, sometimes by different people.
Once you run your work, you want to be able to quickly visualize results, compare them with past results and understand which data was used to produce each model. There are vendor specific solutions for these needs, but you can’t use them if you want to achieve portability across environments.
Iguazio works with leading companies to form a cross platform standard and open implementation for machine learning environments, metadata and artifacts. This allows greater simplicity, automation and portability.
Check out this video to learn how you can move from running/testing code in a local IDE to a production grade automated machine learning pipeline in less than a minute (based on KubeFlow).
