How to Prepare Scikit-Learn Models for Production

Introduction

Data Scientists spend large amounts of effort gathering business requirements, performing exploratory data analysis, data pre-processing, feature engineering, hyperparameter tuning and model evaluation only to have their models stuck in local notebook environments. In order to unlock the full value of the trained models, the models have to be made available to downstream applications. In this article, we walk through the steps to serve scikit-learn machine learning models to downstream applications using Docker and FastAPI. In essence, we will be training a model, wrap the model into an API and containerize the application.

What is Docker?

Docker is an open platform for developing, shipping, and running applications. It provides the ability to package and run an application in a loosely isolated environment called a container. Containers are lightweight and contain everything needed to run the application, so you do not need to rely on what is currently installed on the host [1].

What is FastAPI?

FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.7+ based on standard Python type hints [2].

The content in this article is covered in the following sequence:

Setup
Data
Project directory
Train model
Create API using FastAPI
Test API locally
Create docker Image
Test API in docker container

Setup

Here is the setup used for the walkthrough.

Visual Studio Code
Docker Desktop
Packages

fastapi>=0.68.0,<0.69.0
pydantic>=1.8.0,<2.0.0
uvicorn>=0.15.0,<0.16.0
numpy==1.23.3
scikit-learn==0.23.2
joblib==1.1.0
pandas==1.4.4

Data

We will be using the heart disease dataset [3] for this example. The task is to predict if a patient has a heart disease given various attributes of the patient. condition is the target variable, where 1 indicates presence of heart disease and 0 indicates otherwise.

Project Directory

The project directory structure is as follows:

G:.
│   requirements.txt
│   Dockerfile
│   app.py
|
├───data
│       heart-disease.csv
│
└───model
    │   train.py
    │   heart-disease-v1.joblib

[app.py](<http://app.py>) : contains API logic
/data : the training data (heart-disease.csv) is saved in this directory
/model: the trained model and training script is saved in this directory
[train.py](<http://train.py>) : scikit-learn model is trained in this script
Dockerfile: Dockerfile for containerizing the API
requirements.txt: contains requirements for python packages

Train Model

Filename: train.py

For the purpose of demonstrating FasAPI and Docker, we kept the training simple and did not include any pre-processing or feature engineering. The data is read from /data directory to train a random forest classifier with 10 trees. The trained model is then saved as a .joblib file in /model directory.

#train.py

from os import PathLike
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from joblib import dump
import pandas as pd
import pathlib

df = pd.read_csv(pathlib.Path('data/heart-disease.csv'))
y = df.pop('condition')
X = df

X_train, X_test, y_train, y_test = train_test_split(X,y, test_size = 0.2)

print ('Training model.. ')
clf = RandomForestClassifier(n_estimators = 10,
                             max_depth=2,
                             random_state=0)
clf.fit(X_train, y_train)

print ('Saving model..')
dump(clf, pathlib.Path('model/heart-disease-v1.joblib'))

Create API

Filename: app.py

This is where most of the action happens.

Import libraries

from fastapi import FastAPI
from pydantic import BaseModel
import numpy as np
from joblib import load
import pathlib

Create a FastAPI instance

app = FastAPI(title = 'Heart Disease Prediction')

Load the trained model

model = load(pathlib.Path('model/heart-disease-v1.joblib'))

Define request model

We define the input data required by the API. The format consist of the field name, data type and optional default value.

class InputData(BaseModel):
    age:int=64
    sex:int=1 
    cp:int=3
    trestbps:int=120
    chol:int=267
    fbs:int=0
    restecg:int=0
    thalach:int=99
    exang:int=1
    oldpeak:float=1.8
    slope:int=1
    ca:int=2
    thal:int=2

Define response model

We can define the response data in similar fashion.

class OutputData(BaseModel):
    score:float=0.80318881046519

Define post request

@app.post('/score', response_model = OutputData)
def score(data:InputData):
    model_input = np.array([v for k,v in data.dict().items()]).reshape(1,-1)
    result = model.predict_proba(model_input)[:,-1]

    return {'score':result}

There are many things going on in the above code snippet, let’s break it down.

Let’s examine @app.post('/score', response_model = OutputData) first.

/score is the route name. It is the last part of the URL e.g. "http://my-url.com/score".
@app.post indicates that the /score route accepts a post request.
@app.post('/score') tells FastAPI that the function below (i.e. def score) is in-charge of handling the request that goes to the /score route.
response_model refers to the format data which the API will return to the caller.

Beneath @app.post we have the score function.

The score function requires a data argument which takes the format of InputData which we defined earlier.
The score function then parse the data and convert it into a numpy array
The array is passed to the .predict_proba method which returns a score between 0-1.

Run the live server

We can launch the API locally in the following manner. In the terminal, at the project root directory run:

uvicorn app:app --reload

The output of a successful deployment looks like this:

INFO:     Will watch for changes in these directories: ['path/to/project']
INFO:     Uvicorn running on <http://127.0.0.1:8000> (Press CTRL+C to quit)
INFO:     Started reloader process [24040] using StatReload
INFO:     Started server process [26612]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

The second line in the above code block shows the URL where the app is being served.

Automated API documentation

Based on the API definitions, FastAPI generates automated documentation in Swagger and Redoc UI. Access the Swagger API documentation using http://127.0.0.1:8000/docs.

and the Redoc API documentation using http://127.0.0.1:8000/redoc.

Test API Locally

At this point, the API can be tested either through swagger UI, terminal or using python.

Swagger UI

Terminal

We can call the API through the terminal using curl. The data argument required by the score function is passed through the -d argument in curl.

curl -X 'POST' 
  '<http://127.0.0.1:8000/score>' 
  -H 'accept: application/json' 
  -H 'Content-Type: application/json' 
  -d '{
  "age": 64,
  "sex": 1,
  "cp": 3,
  "trestbps": 120,
  "chol": 267,
  "fbs": 0,
  "restecg": 0,
  "thalach": 99,
  "exang": 1,
  "oldpeak": 1.8,
  "slope": 1,
  "ca": 2,
  "thal": 2
}'

Python

The API can be called using python’s requests package. The data argument required by the score function is passed through the json argument in request.post.

import requests

body = {
    "age": 64,
    "sex": 1,
    "cp": 3,
    "trestbps": 120,
    "chol": 267,
    "fbs": 0,
    "restecg": 0,
    "thalach": 99,
    "exang": 1,
    "oldpeak": 1.8,
    "slope": 1,
    "ca": 2,
    "thal": 2
    }

response = requests.post(url = '<http://127.0.0.1:8000/score>',
              json = body)

print (response.json())
# output: {'score': 0.866490130600765}

Create Docker Image

Great! The API works locally. Often we may be required to deploy the API in cloud environment such as AWS, Azure or GCP. One of the ways to do so is by containerizing your application.

Define python requirements

Let’s define the requirements in requirements.txt

# requirements.txt
fastapi>=0.68.0,<0.69.0
pydantic>=1.8.0,<2.0.0
uvicorn>=0.15.0,<0.16.0
numpy==1.23.3
scikit-learn==0.23.2
joblib==1.1.0

Define Dockerfile

#Dockerfile
FROM python:3.8

WORKDIR /app

COPY ./requirements.txt ./requirements.txt

RUN pip install --no-cache-dir --upgrade -r ./requirements.txt

COPY ./app.py .
COPY ./model ./model

CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "80"]

The Dockerfile does the following:

Get a python 3.8 image from Docker hub
Create a work directory called /app
Copy requirements.txt into the work directory
Install packages defined in requirements.txt
Copy the code and model into work directory
CMD launches the server on port 80 when the container is started

Build Docker image

# at the project root folder
docker build -t fastapiml .

Test In Docker Container

Launch Docker container

docker run -d --name heart-disease-serving -p 80:80 fastapiml

The running container will be shown in Docker Desktop.

Automated API Documentation

Similar as before, we are able to access the API documentations using:

Swagger: [http://127.0.0.1/docs](http://127.0.0.1/docs)
Redoc: [http://127.0.0.1/redoc](http://127.0.0.1/redoc)

The API running in docker container can be tested in similar fashion as before. Either using Swagger UI, curl command at the terminal or python’s requests package.

Conclusion

In this article we discussed the importance of making machine learning models accessible to downstream applications. We can unlock the value of machine learning models by serving these models through APIs using FastAPI. One of the advantages of using FastAPI is its auto documentation in Swagger which also allows quick tests to be conducted. We also demonstrated how to containerize FastAPI applications which allows for easier deployment to the cloud.

Join Medium to read more stories like this!