
Introduction
Serverless container architecture is an approach to building and running containerized applications and services without having to manage the underlying infrastructure. In this architecture, containers are used to package and deploy applications, and these containers are run in a fully managed environment provided by a cloud provider.
The cloud provider is responsible for the infrastructure needed to run the containers, such as the hardware and operating system. The developer does not need to worry about setting up or maintaining this infrastructure, and can instead focus on writing code and building applications. The containers are typically run in a cluster, and the cloud provider automatically scales the number of containers up or down based on demand. This allows the application to handle fluctuations in traffic without the need for manual intervention. Serverless architectures can be more cost-effective than traditional architectures, since users only pay for the resources that was actually use, rather than paying for a fixed amount of computing power that may or may not be fully utilized.
Some examples of serverless container services include Azure Functions, Azure Container Apps , AWS Lambda, and Google Cloud Functions. In this article we will demonstrate how to leverage on Azure Container Apps, a fully managed serverless container service for building and deploying apps at scale, for productionizing machine learning models. Common uses of Azure Container Apps include deploying API endpoints, hosting background processing applications, handling event-driven processing and running microservices [2].
These are the steps that we will take in order to train and deploy a scikit-learn model with Azure Container Apps.
- Train model locally
- Create inference API with FastAPI
- Dockerize the application
- Deploy to Azure Container Apps
0. Setup
This is the setup used for the following example.
Development Environment
- Visual Studio Code
- Azure CLI
- Python 3.8
- Docker
- Python Packages: Refer to Section 3 for
requirements.txt
Project Structure
The project folder structure as follows:
FastAPI-Azure-Container-Apps
├─ fastapp
│ └─ app.py
├─ model
│ ├─ heart-disease.joblib
│ └─ train.py
├─ data
│ └─ heart-disease.csv
├─ Dockerfile
├─ requirements.txt
└─ .dockerignore
Dataset
The UCI Heart Disease dataset [3] is a public dataset that contains data about patients who have been diagnosed with heart disease. It includes various patient characteristics, such as age, gender, blood pressure, and cholesterol levels. 1
and 0
values in the condition
column represents the presence or absence of heart disease respectively.
1. Train model locally
File: train.py
For the purpose of illustration, we will train a Gradient Boosting Classifier using only 5 features.
import pathlib
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import GradientBoostingClassifier
from joblib import dump
print ('Load Data')
df = pd.read_csv(pathlib.Path('data/heart-disease.csv'))
y = df.pop('condition')
X = df.loc[:, ['age', 'sex', 'cp', 'trestbps', 'chol']].to_numpy()
print ('Train-Test Split')
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size = 0.2)
print ('Train Model')
gbc = GradientBoostingClassifier()
gbc.fit(X_train, y_train)
print ('Save Model')
dump(gbc, pathlib.Path('model/heart-disease.joblib'))
Run the training using CLI:
python model/train.py
2. Create prediction endpoint with FastAPI
File: app.py
from fastapi import FastAPI
from pydantic import BaseModel
import numpy as np
from joblib import load
import pathlib
app = FastAPI(title = 'Heart Disesase Prediction', version = '0.1.0')
model = load(pathlib.Path('model/heart-disease.joblib'))
class ModelData(BaseModel):
age:int=30
sex:int=0
cp:int=2
trestbps:int=115
chol:int=0
class ResponseData(BaseModel):
prediction_result:float=0.1
@app.post('/predict', response_model = ResponseData)
def predict(data:ModelData):
input_data = np.array([v for k,v in data.dict().items()]).reshape(1,-1)
prediction_result = model.predict_proba(input_data)[:,-1]
return {'prediction_result':prediction_result}
In the app.py
file we
- Define
app
, an instance of FastAPI. - Load the trained model.
- Define the input data format (
ModelData
) which the API will accept - Define the API response format (
ResponseData
) - Define the
/predict
route which will trigger thepredict
function when a post request is made to this route. - The
predict
function receives the input data from the post request, predict and returns the probability that a patient has heart disease.
At this point we can test the FastAPI application locally. The --reload
flag helps with speeding up the development process. FastAPI auto reloads every time a change in the code is detected. This means that the developer does not need to manually restart FastAPI to test the code changes.
# CLI
uvicorn fastapp.app:app --reload
You will see the following message on the terminal:
INFO: Uvicorn running on <http://127.0.0.1:8000> (Press CTRL+C to quit)
The given URL will bring us to the Swagger UI where we can test out the API.
3. Containerize the application using Docker
Create a requirements.txt
file
The requirements.txt
file contains all the python packages required.
fastapi>=0.68.0,<0.69.0
pydantic>=1.8.0,<2.0.0
uvicorn>=0.15.0,<0.16.0
numpy == 1.19.5
scikit-learn==0.23.2
joblib==1.1.0
nest_asyncio == 1.5.5
Create a .dockerignore
file
The purpose of the .dockerignore
file is to avoid copying over files that are not required for inference such as the training script and data.
data/
model/train.py
Create a Dockerfile
file
FROM python:3.8
WORKDIR /app
COPY . /app
RUN pip install -r requirements.txt
CMD ["uvicorn", "fastapp.app:app", "--host", "0.0.0.0", "--port", "80"]
Here’s a brief description of the Dockerfile:
- Use
python:3.8
as the base image - Creat a working directory named
/app
- Copy all the files in our project folder over to the working directory with the exception of those files or sub-directories that were listed in the
.dockerignore
file. - Install the python packages listed in
requirements.txt
CMD
runs the FastAPI application on port 80 when the docker container is launched. Unlike in the local test, we do not include – reload flag when we run uvicorn here. The reload flag is useful for speeding up the development process, however it is not required in production.
Build docker image
docker build . -t heart-disease-app
Launch docker container
We map port 80 in the container, which FastAPI is running on, to port 8080 on the Docker host.
docker run -p 8080:80 -it heart-disease-app
Test the app
At this point we can test the application through the Swagger UI again by going to the URL: [http://127.0.0.1:8080/docs](http://127.0.0.1:8080/docs)
4. Deploy to Azure Container Apps
In this section, we will push the docker image to Azure Container Registry and subsequently deploy the docker container in Azure Container Apps.
To deploy the containerized application to Azure Container Apps we require the following prerequisites:
- Azure Resource Group
- Azure Container Registry
- Azure Container App Environment
The following commands are executed in the command line.
Create resource group
A resource group is a logical grouping of Azure services used to support the application. We created a heartdisease_rg
resource group in eastus
location. All the subsequent resources will be assigned toheartdisease_rg
.
az group create --location eastus --name heartdisease_rg
Create Azure Container Registry
Azure Container Registry (ACR) is a collection of repositories used to store and manage container images. We create a container registry named heartdisease
under the heartdisease_rg
resource group and chose the Basic
SKU pricing plan.
az acr create --name heartdisease --resource-group heartdisease_rg --sku Basic
Once the container registry is provisioned we can check the ACR login server using
az acr list --resource-group heartdisease_rg
The above command returns a long string which contains the login server. Take note of the login server details as we will be using it in the next step.
...
"location": "eastus",
"loginServer": "heartdisease.azurecr.io",
"name": "heartdisease"
...
Tag docker image
To push the local docker image to ACR, the docker image heart-disease-app
is tagged the following format {login server}/{docker image name}/{version}
.
docker tag heart-disease-app heartdisease.azurecr.io/heart-disease-app:v0.1.0
Login to ACR
Ensure that you are logged in before pushing image to ACR.
az acr login -n heartdisease
Push docker image to ACR
Docker push is a command that uploads a local docker image to a container registry. This will make the docker image available to Azure Container Apps.
docker push heartdisease.azurecr.io/heart-disease-app:v0.1.0
The docker image will be shown in ACR’s UI upon successful docker push.

Create Azure Container App Environment
Before creating Azure Container Apps, we define the heartdiseaseenv
environment which the Container App will run in.
az containerapp env create --name heartdiseaseenv --resource-group heartdisease_rg --location eastus
Create Azure Container Apps
In this step, we create the heart-disease-container-app
Azure Container Apps using the heartdiseaseenv
environment which we created in the step before. We also defined the docker image that should be used: heartdisease.azurecr.io/heart-disease-app:v0.1.0
and the login server of the container registry: heartdisease.azurecr.io
. The ingress
is set to external
as this API is meant to be exposed to the public web.
az containerapp create --name heart-disease-container-app --resource-group heartdisease_rg --environment heartdiseaseenv --image heartdisease.azurecr.io/heart-disease-app:v0.1.0 --target-port 80 --ingress external --query properties.configuration.ingress.fqdn --registry-identity system --registry-server heartdisease.azurecr.io
If az containerapp create
command is successful, it will return a URL for accessing your app.
Container app created. Access your app at <https://heart-disease-container-app.nicehill-f0509673.eastus.azurecontainerapps.io/>
Test the app
We can test the app using either the Swagger UI, curl or python request. To access the Swagger UI, simply append docs
at the end of the given URL: https://heart-disease-container-app.nicehill-f0509673.eastus.azurecontainerapps.io/docs
.
To use CURL, the command as follows:
curl -X 'POST'
'<https://heart-disease-container-app.nicehill-f0509673.eastus.azurecontainerapps.io/predict>'
-H 'accept: application/json'
-H 'Content-Type: application/json'
-d '{
"age": 64,
"sex": 1,
"cp": 3,
"trestbps": 120,
"chol": 267
}'
We can also send a post request to the prediction endpoint https://heart-disease-container-app.nicehill-f0509673.eastus.azurecontainerapps.io/predict
using python’s request library in the following manner:
import requests
response = requests.post(url = '<https://heart-disease-container-app.nicehill-f0509673.eastus.azurecontainerapps.io/predict>',
json = {"age": 64,
"sex": 1,
"cp": 3,
"trestbps": 120,
"chol": 267
)
print (response.json())
# {'prediction_result': 0.8298846604381431}
Conclusion
In this article, we discussed the advantage of using serverless containerized Machine Learning inference endpoint and walked through an example of how to create an API endpoint with FastAPI, containerized it with Docker and deployed the containerized application with Azure Container Apps.
Join medium to read more articles like this!
Reference
[1] Serverless computing and applications | Microsoft Azure
[2] Azure Container Apps overview | Microsoft Learn
[3] Heart Disease Dataset from UCI Machine Learning Repository. Licensed under CC BY 4.0.