
We are going to see that compared to previous articles,,) this is super easy to create a Kubernetes cluster with IBM Cloud. The wealth of Kubernetes resources can make it difficult to find the basics. An easy way to simplify Kubernetes development and make it easy to deploy is to use solutions such as IBM Cloud Kubernetes Services. In order to create a machine learning application that deploys to the IBM Cloud Kubernetes Service, we need an IBM Cloud account (sign up for a free account), IBM Cloud CLI, Docker CLI, Kubernetes CLI.
Create Kubernetes service on IBM Cloud
The Kubernetes service on IBM Cloud provides two cluster types:
- a free cluster (one worker pool with a single virtual-shared worker node with 2 cores, 4GB RAM, and 100GB SAN)
- a fully customizable standard cluster (virtual-shared, virtual-dedicated, or bare metal) for the heavy lifting.
If we just want to explore, the free cluster is great.
In IBM Cloud, with few clicks, we can automatically create a Kubernetes service. First, we need to connect to our IBM Cloud Dashboard.
We go to IBM Kubernetes Service, click on create clusters, type in a name for our cluster, depending on our account (paid or free), we can select the appropriate cluster type (in our case we will just create a worker node with 2 vCPUs and 4 Go of RAM), and after a few minutes, the cluster is created.

Once the cluster is ready, we can click on our cluster’s name, and we will be redirected to a new page with information about our cluster and worker node.

To connect to our cluster, we can click on our worker node tab to get the public IP of the cluster.

Done! We can have a quick access using the IBM Cloud Shell.
If we want to use our own terminal, we need some prerequisite if not already installed. We need to install the required CLI tools: IBM Cloud CLI, Kubernetes Service plug-in (ibmcloud ks), Kubernetes CLI (kubectl).
To install the IBM Cloud CLI, we will type in our terminal the following to install the stand-alone IBM Cloud CLI (ibmcloud):
curl -fsSL https://clis.cloud.ibm.com/install/linux | sh
This is for Linux. You can find all necessary commands for your own distribution.
We log in to the IBM Cloud CLI by entering our IBM Cloud credentials when prompted:
ibmcloud login
If we have a federated ID, we can use ibmcloud login –sso to log in to the IBM Cloud CLI.
Otherwise, we can also connect with an IBM Cloud API key as follows:
ibmcloud login - apikey < IBM CLOUD API KEY >
If not already done, we can create an IBM Cloud API key. To do this, we need to go to the IBM Cloud console, go to Manage > Access (IAM) and select API keys:

We can click create an IBM Cloud API key, add a name and description, and copy or download the API key to a secure location. We can then login thanks to the command above.
We can install the IBM Cloud plug-in for IBM Cloud Kubernetes Service (ibmcloud ks):
ibmcloud plugin install container-service
Install the IBM Cloud plug-in for IBM Cloud Container Registry (ibmcloud cr):
ibmcloud plugin install container-registry
We can also install the IBM Cloud Kubernetes Service observability plug-in (ibmcloud ob)
ibmcloud plugin install observe-service
Kubernetes CLI is already installed in our environment. If not already install, just follow few steps here.
If we want to list all the clusters in the account:
ibmcloud ks cluster ls

We can check if our cluster is in a healthy state by running the following command:
ibmcloud ks cluster get -c IBM_Cloud_node
Here, _IBM_Cloudnode is our cluster name; you can also use the ID of the cluster.

Containerization of a Machine Learning application
This quick example will show how to create a Docker container to perform online inference with a trained machine learning model using Python API with Flask. To do that, we will train a simple C-Support Vector Classification model using scikit-learn and Iris dataset that we will split into train data and test data.
To start, let’s consider the following files:
- Dockerfile
- train.py
- api.py
- requirements.txt
You can find all the files on GitHub.
The train.py is a python script that loads and trains our model. The Dockerfile will be used to build our Docker image, requirements.txt (flask, flask-restful, joblib) is for the Python dependencies and api.py is the script that will be called to perform the online inference using an API.
The train.py file is the following:
We also need to build an API that will ingest the data (X_test) and output what we want. In our case, we will just request the classification score of the model:
We now are ready to containerize your Flask application. In our project directory, we created our Dockerfile with jupyter/scipy-notebook image as our base image, set our environment variables and install joblib and flask, we copy train.py and api.py files into the image.
We want to expose the port (5000) the Flask application runs on, so we use EXPOSE.
In order to check if our application is running without issue, let’s build and run our image locally:
docker build -t my-kube-api -f Dockerfile .

docker run -it -p 5000:5000 my-kube-api python3 api.py

We can now test the application using curl:
curl http://172.17.0.2:5000/score

Everything is working fine.
Push the image to the IBM Cloud Registry
It works! Now that our application is working properly, we can move to the next step and deploy it in a Kubernetes Cluster. Before doing that, we need to push the image to a repository. Here, we will push the image on the IBM Cloud Registry (a private repository). From our account dashboard, we can select Container Registry:

We need to install the Container Registry plug-in locally by using the following command:
ibmcloud plugin install container-registry -r "IBM Cloud"

Then, we login into our account:
ibmcloud login
Then, we name and create our namespace:
ibmcloud cr namespace-add xaviervasques

We log our local Docker daemon into the IBM Cloud Container Registry using the following command:
docker login -u iamapikey -p <YOUR API KEY> de.icr.io
We choose a repository and tag by which we can identify the image:
docker tag my-kube-api de.icr.io/xaviervasques/my-kube-api:latest
And we push the image (docker push //:):
docker push de.icr.io/xaviervasques/my-kube-api:latest

We can verify the status of our image by checking if it’s on our private registry:
ibmcloud cr image-list
Deploy the application to Kubernetes
Once the image is uploaded to the private registry, we can deploy the application to Kubernetes. We can use the user interface or the CLI. For this chapter, we will use CLI. We create our Kubernetes cluster using the steps above (we can also create one using command line: ibmcloud ks cluster create classic –name my-cluster). To see the status, we type the following command:
ibmcloud ks clusters

Our my-k8s Kubernetes cluster is up and running. We can connect kubectl to the cluster:
ibmcloud ks cluster config - cluster my_k8s
We can check if we are connected to the cluster:
kubectl get nodes

We will create a folder named "base" in the master node and create the following YAML files inside:
o namespace.yaml
o deployment.yaml
o service.yaml
o service_port.yaml
o kustomization.yaml
The namespace.yaml file provides a scope for Kubernetes resources:
The deployment.yaml will lets us manage a set of identical pods. If we do not use a deployment, we would need to create, update and delete a bunch of Pods manually. It is also a way to easily autoscale our applications. In our example, we decided to create two Pods (replicas), load our Docker image that we pushed previously and run our api.py script.
The service.yaml file will expose our application running on a set of Pods as a network service.
We also need to create the service_port.yaml file :
The reason we create the service_port.yaml file is to make our containerized app accessible over the internet by using the public IP address of any worker node in a Kubernetes cluster and exposing a node port (NodePort). We can use this option for testing IBM Cloud Kubernetes Service and for short-term public access.
Finally, we create the kustomization.yaml file:
We can configure our own image pull secret to deploy containers in Kubernetes namespaces other than the default namespace. With this methodology, we can use images stored in other IBM Cloud accounts, or us images stored in external private registries. In addition, we can create our own image pull secret to enforce IAM access rules that restrict rights to specific registry image namespaces or actions (such as push or pull). We have several options to do that, one of them is to copy the image fetch secret from the Kubernetes default namespace to other namespaces in our cluster .
Let’s start by listing the namespaces in our cluster:
kubectl get namespaces

Then, let’s list the image pull secrets in the Kubernetes default namespaces for IBM Cloud Container Registry:
kubectl get secrets -n default | grep icr-io

To deploy our application, we use this single command in our master node:
kubectl apply - kustomize=${PWD}/base/ - record=true
We copy the all-icr-io image extraction secret from the default namespace to the namespace of our choice. The new image fetch secrets are named -icr- -io :
kubectl get secret all-icr-io -n default -o yaml | sed 's/default/mlapi/g' | kubectl create -n mlapi -f -
We check that the creation of secrets was successful:
kubectl get secrets -n mlapi | grep icr-io
To see all components deployed into this namespace:
kubectl get ns
We should obtain the following output:

To see the status of the deployment, we can use the following command:
kubectl get deployment -n mlapi

To see the status of the service, we use this command:
kubectl get service -n mlapi

We can obtain the public IP address of a worker node in the cluster. If you want to access the worker node on a private network or if you have a VPC cluster, obtain the private IP address instead.
ibmcloud ks worker ls - cluster my_k8s
We are now ready to use our deployed model by using curl or your web browser:
curl http://172.21.193.80:31261/score

We also navigate through our Kubernetes Dashboard and check our services and many features:

Next steps
When we work on putting Machine Learning / deep learning models into production, there is a question coming at some point in time. Where I deploy my code for training, where I deploy my code for batch or online inference. And it can often happen that we need to deploy our machine learning flow on a multi-architecture environment and hybrid/multi cloud environment. We have seen how to deploy an app on IBM Cloud and how to deploy on-premise / virtual machines. Kubernetes can run on a variety of platforms: from simple clusters to complex ones, from our laptop to multi-architectures, hybrid / multi cloud Kubernetes Clusters. The question is what is the solution that best suits our needs.
Sources
https://developer.ibm.com/technologies/containers/tutorials/scalable-python-app-with-kubernetes/
https://cloud.google.com/community/tutorials/kubernetes-ml-ops
https://github.com/IBM/deploy-ibm-cloud-private
https://kubernetes.io/fr/docs/setup/pick-right-solution/#solutions-clés-en-main
https://www.ibm.com/cloud/architecture/tutorials/microservices-app-on-kubernetes?task=1
https://cloud.ibm.com/docs/containers?topic=containers-registry#other
https://cloud.ibm.com/docs/containers?topic=containers-nodeport