In the previous article, we explored serving our tensorflow models using tensorflow-serving and how we can use Google Cloud to deploy our models in production using Docker and Kubernetes. If you haven’t explored the part-1 of this series I suggest you do, to get familiar with these terminologies and have a better understanding for following this guide.
In this article, we will focus on AI-Platform to deploy our model and then consume it in our application. AI Platform is basically a complete set of services that caters the whole ML-Pipeline from ingesting data till Deployment. It covers data pre-processing, validating data, building ML-models, and finally deploying them, just to add further it doesn’t stop here, it also offers MLOps for implementing CI/CD for Machine-Learning Systems. Some of the key features of AI-Platform include
- Highly scalable
- Low Latency
- Powered by Tensorflow-Serving
- Support for multiple models & versions
- No infrastructure management
- Production-ready.
Now that we have a basic understanding of AI-Platform let’s dive into deploying our model on AI-Platform, we will be using the same mnist-digit-classifier that we developed in the previous post. You can of course use your own Machine Learning models, but for the sake of this article, I will be sticking with mnist-digit-classifier. Before moving on you need to configure a new Google-Cloud account if you haven’t already and export your model.
Let’s get started, there are a couple of steps in order to deploy our model, each of these can either be performed via the Cloud-Console or the Clouds-SDK.
Uploading your Model to a Cloud Storage Bucket
First, we need to upload our model to a cloud storage bucket, for this, you can simply head over to your GCP-project and navigate to the storage service. From there you can create a bucket and give it a meaningful name, I have called mine mnist-digit.

Now that our bucket is created we can simply upload our model to the bucket using the below command.
gsutil cp -r <path-to-tf-js-model-dir> gs://<bucket-name>/
Please note that a good convention for organizing your models is to use a sub-folder in your model’s directory which highlights the model version that can be seen below.

If you re-train your model or fine-tune the model architecture and export a different version you can create a separate folder here named 2 which will contain the updated model. The contents of this directory will be the entire saved model-directory as seen below.

Note: Your bucket must be in the same region as the AI-platform otherwise you will have to explicitly grant access to the AI-Platform.
Perfect we are done with the first step in getting our model uploaded to a cloud storage bucket, next comes
Creating an AI-Platform Model Resource
Before deploying the model first head over to your GCP project APIs & Services → Enable APIs & Services → AI Platform Training & Prediction API and enable this API.

Next head over to the AI-Platform service from the left side-bar in GCP and navigate to the Models tab.

Click on New model to create a model resource and give your model a unique name within the project, select a region this will be the region where the jobs will be running, once selected hit Create. I have called mine mnist_digit_recog

Now if you go to the Models tab you should see your model resource. Next, we will be creating a version resource for our model resource that we just created.
Creating an AI-Platform Version Resource
Now we will be creating a version with the trained model we uploaded to the cloud-storage bucket in the first step. For this click on the model resource we just created and hit create a version.

Next, select a version for your model and set the python version and framework. I trained my model on python version 3.7.4. To check your python version you can run the below command in terminal
python --version (for python-2)
python3 --version (for python-3)
To check TensorFlow version you can run
import tensorflow as tf
tf.__version__
Next, configure the path to your cloud storage bucket where you uploaded the model make sure you configure the path to the folder containing the model files which in our case was the directory named "1".

Once you have configured it click save, this will take up some time and once it’s finished you should see a green checkmark next to your deployed version as shown below

Now we are all set to make predictions with our deployed instance. Btw if you are a fan of command-line then all these steps can be performed via the google cloud sdk. To explore the command line options please visit the official-docs.
Inference using AI-Platform
Now that we have our model deployed let’s make some inferences, click on your model version and you will be taken to this page with additional details about the model and traffic.

Here we can see that our deployed model is expecting an object with an instances **** key containing the test data. To learn more about getting online predictions do give this link a visit which highlights how you should input your data and get predictions in response.
We will be using the google api client for python to get inference from our deployed model. If you are working in some other programming-language you can check out this page to see if the language is supported.
To get predictions we first need access to make calls to the model for which we need a credentials file. To set up the credentials in google cloud console, head over to APIs & Services and go to the credentials tab. From there click create new credentials and select service account. Give your service account a name and assign an appropriate role. I assigned my service account ML Engine Admin and Storage Object Admin. Once done click create.

Once created, click your service account and scroll down to Add Key click the button and select create a new key and set the format as JSON, and hit create.
Download the JSON on disk and rename it for your own convenience. I have renamed it as [project-name]_credentials.json.
Next we will install the python client libraries using the below command
pip3 install google-api-python-client oauth2client
For more details you can visit the official docs for python client here
Once installed we can use the below code snippet to request our model.
Note that I am specifying the api_endpoint
with the specific region in which I created my model resource. You can check that out quickly by going to your model resource in AI-Platform.

And that’s it now you can deploy your machine learning models on AI-Platform whether they were trained using AI-Platform or not, and consume them from any application. In the last article of this series, we will look at how we can use cloud-functions for serving our models.
If you find this article useful do share it in your circle and leave a clap. For any queries or suggestions, you can leave a comment or you can connect with me on LinkedIn.
Happy Coding.