The world’s leading publication for data science, AI, and ML professionals.

Quick and Easy Deep Learning WebApp

Train and deploy a deep learning image classification model with fastai and streamlit

Photo by Sid Balachandran on Unsplash
Photo by Sid Balachandran on Unsplash

Have you ever wanted to use deep learning in a project and show your results in an interactive way? Do you want to start your deep learning journey and quickly see your results as a working application? Let me show you how.

One of the easiest ways to deploy a small data-driven or machine learning app with Python is streamlit. Streamlit allows you to create WebApps with a lot of functionality using just a few lines of code as I will demonstrate in a minute.

Streamlit – The fastest way to create data apps

Now that covers the deployment part, but what about the machine or deep learning? If you want to dive into deep learning, fast.AI is a great place to start. Their MOOC, now also their book and especially the fastai library provide you with everything you need to start your deep learning journey.

Home

I want to show you just how quick & easy you can build an interactive classification WebApp that uses state of the art deep learning models. To do that I will take the 225 Bird Species Dataset from Kaggle, train a deep learning model on it and then use that model in a streamlit App.

Photo by Dulcey Lima on Unsplash
Photo by Dulcey Lima on Unsplash

Now I realize that classifying that many bird species is technically not the easiest task, and the model is far from perfect, if you want really good results you will have to dig a bit deeper. I just want to provide you with a classification app tutorial that doesn’t use MNIST, Fashion MNIST, CIFAR-10 or anything the Fastai MOOC already uses. This tutorial is not showing you how to get a top spot on the Kaggle leaderboard for bird image classification. I’d rather show you how to build something end-to-end in a beginner friendly manner.

The first step is to download the data. The Dataset contains 4 folders named "consolidated", "test", "train" and "valid". Within each folder we can find 225 subfolders, one for each species. The folder names will provide the labels later. The "consolidated" folder contains all images, so it’s only of limited use for us.

Let’s focus on the train and validation folder for our task.

Data Prep & Model training

The next step is to load our data, preprocess it and train our model. This may sound like a lot and possibly overwhelming if you are a beginner, but fastai makes it surprisingly simple.

We first import the fastai.vision.all package. Please note that I am using import * here. I know about the issues this can create, I don’t recommend using it for anything other than exploration and experimentation within a jupyter notebook. Once we’ve explored, trained and saved the model we will not use this kind of import again in our WebApp file.

from fastai.vision.all import *
#set the path where our data can be found
path = Path("D:/datasets/birds/archive (2)")
#check if the path is working by extracting one image
im = Image.open(path / "train/CROW/001.jpg")
im.to_thumb(224,224) #this should display your image

Now we can build a DataBlock and dataloader to load the data in a way our model can process them. In fastai this is done through the DataBlock class and the dataloaders. The GrandparentSplitter() tells the DataBlock to use all files in "train" for the training set and all in "valid" for the validation set.

birds = DataBlock(
        blocks = (ImageBlock,CategoryBlock),
        get_items = get_image_files,
        splitter = GrandparentSplitter(train_name = "train", valid_name = "valid"),
        get_y = parent_label, #get labels from folder names
        item_tfms = Resize(224) #resize images to 224x224
)
#use a dataloader to get the images
dls = birds.dataloaders(path)

This would be a good moment to check if everything went as expected. We will look into the first 10 pictures of the validation and training data to see if everything has been loaded correctly.

dls.valid.show_batch(max_n = 10, nrows = 2)
dls.train.show_batch(max_n = 10, nrows = 2)

Your output should look something like this:

Image by Author.
Image by Author.

Note that the training data is already shuffled for training, while the validation data is not and also doesn’t need to be.

Most of the pictures our classifier will have to deal with in a real setting won’t be perfect images worthy of a National Geographic feature (i.e. when someone wants to use your app to find out which bird they’ve taken a photo of). We should therefore add some transformations to the data. With birds.new() we can add new images or transformed images in our case. Using RandomResizedCrop we can add pictures which are random crops from the original picture, with batch_tfms = aug_transforms() we add transformations to the whole batch, these transformations are some useful standard image augmentations such as changing the brightness, tilting, stretching and squishing the picture.

birds = birds.new(
        item_tfms = RandomResizedCrop(224, min_scale = 0.5),
        batch_tfms = aug_transforms())
#we again load the data 
#I set num_workers = 0 in order to run the torch code on my local #machine, this may not be necessary for you
dls = birds.dataloaders(path, num_workers = 0)
#look at some of the transformations
dls.train.show_batch(max_n = 15, nrows = 3, unique = True)

Some of these pictures may look a little odd, but you can also spot some "okay but far from ideal" pictures, which may resemble something a user of your WebApp would upload later.

Okay, so we have our data, we preprocessed it a bit. It’s time to choose a model and train it.

Fastai strongly emphasizes using pretrained models and fine tuning them. This often leads to better and faster results than training something from scratch. Here we will use a ResNet34 which was pretrained on the ImageNet dataset. Hence, our model has already "seen" pictures and also kind of "knows" what a bird looks like. We can use that and now fine tune the model towards classifying different species of birds.

learn = cnn_learner(dls, resnet34, metrics=error_rate).to_fp16() 
#to_fp16() works well for my local system, might not be necessary for you
#lets train and tune the model for 5 epochs
learn.fine_tune(5)

For an in-depth explanation of the fine_tune() method see the fastai docs.

Once the model is trained we can check which kind of errors it makes, look into it’s loss over training time and of course use the test set to see how well it does on new data. You should definitely do all of these things. If you don’t know how, check the fast.ai tutorials or the MOOC.

However, I am not discussing these steps here as the main focus is not the model training and evaluation, but showing how to create a working app.

The model did pretty well. The error rate (on the validation set) is 0.02 for me, which translate into 98% accuracy. Please note that accuracy is not always the best metric and if you want to train a model you should look into more metrics.

It’s time to save our model and move into the next stage: Building the WebApp.

learn.export(fname = "first_model.pkl")

You should probably find a more meaningful name, especially if you have multiple models.

The Streamlit App

We are going to use streamlit as mentioned. To host your application locally open your terminal or command/anaconda prompt navigate to the directory your Python file is located and type streamlit run {your_file_name}.py.

The app should be available under localhost:8501.

We will need streamlit, numpy, PIL and fastai.vision to run everything.

import streamlit as st
import numpy as np
from PIL import Image
from fastai.vision.all import load_learner, Path

The beginning of our App should have title. Which in streamlit is as easy as typing st.title("Your chosen title")

The next part of our App is the file upload which is quick and convenient with: uploaded_file = st.file_uploader("Choose an image...", type="jpg")

st.title("Upload + Classification Example")
uploaded_file = st.file_uploader("Choose an image...", type="jpg") #file upload

We now basically have finished the first state of the app, which looks like this:

Image by Author
Image by Author

To integrate our classification we first need to load the model.

learn_inf = load_learner(Path("first_model.pkl"))#load trained model

Now let’s classify the image and create some output.

The code checks whether there is an uploaded file, and once that is true the image file is converted in order to let our model predict its label using the .predict() method.

Once we have a prediction we create some output for our user with st.write() which uses standard Markdown formatting. I also added a small if statement to have better grammar.

#classification
if uploaded_file is not None:
    #image transformation and prediciton
    img = Image.open(uploaded_file)
    st.image(img, caption='Your Image.', use_column_width=True)
    image = np.asarray(img)
    label = learn_inf.predict(image) 
    #label[0] accesses the actual label string
    #output display
    st.write("")
    st.write("Classifying...")
    #check for vowels in the names for correct grammar
    if label[0][0] in "AEIOU":
        st.write("## This looks like an")
    else:
        st.write("## This looks like a")
    #our labels are in capital letters only
    st.title(label[0].lower().title())

After you’ve uploaded an image the WebApp should look somewhat like this:

Great! Now we have a functioning, local WebApp that uses our deep learning model! Much cooler than reading through a jupyter notebook!

You could deploy this, however you may need to make some changes, regarding the use of GPUs etc.

Here is the whole code for the App:

###imports
import streamlit as st
import numpy as np
from PIL import Image
from fastai.vision.all import load_learner, Path
## You may need these to deploy your model on e.g. heroku if you are coming from a windows machine
#import pathlib
#temp = pathlib.WindowsPath
#pathlib.WindowsPath = pathlib.PosixPath
###App
##file upload and model loading
st.title("Upload + Classification Example")
uploaded_file = st.file_uploader("Choose an image...", type="jpg") #file upload
learn_inf = load_learner(Path("first_model.pkl"))#load trained model
#learn_inf.model.cpu() # use this if you want to deploy the model somewhere without using a GPU, you may need pytorch cpu
##classification
if uploaded_file is not None:
    #image transformation and prediction
    img = Image.open(uploaded_file)
    st.image(img, caption='Your Image.', use_column_width=True)
    image = np.asarray(img)
    label = learn_inf.predict(image)
    #label[0] accesses the actual label string
    #output display
    st.write("")
    st.write("Classifying...")
    #check for vowels in the names for correct grammar
    if label[0][0] in "AEIOU":
        st.write("## This looks like an")
    else:
        st.write("## This looks like a")
    #our labels are in capital letters only
    st.title(label[0].lower().title())

You now know how to build an Image Classification App based on Deep Learning!

Streamlit offers many features and you could add more functionality or display more data or change the layout a bit, experiment and explore. Fastai also offers tons of features, which I recommend you to explore as well.

I hope this article was helpful to you and now I ask you to:

Explore, experiment, build your own Apps and showcase what you’ve learned!

-Merlin


Related Articles