Deploy Containerized Plotly Dash App with CI/CD (P1: Heroku)

Photo by Bernd Dittrich on Unsplash

Say you have locally developed a cool application that you’d like to share with the world. In addition, once your app is live, you want to easily make changes that are directly reflected online. This can be a daunting task involving the selection of a cloud provider, containerizing your app and automating the deployment pipeline. If this seems like a daunting task, then please read on, as this article provides a breakdown of each component.

Edit: I have changed the deployment pipeline to deploy to Google Cloud Run, for a full overview of deploying through CI/CD to Google Cloud Run please have a look at P2.

More specifically, this article is a step-by-step guide on how to build and deploy a containerized Plotly Dash app on Heroku using Github actions for CI/CD. Note that the Dash and Heroku parts can be swapped by other app frameworks and other cloud platforms while the overall structure remains the same. By the end of this guide you’ll only need to push a change to the main branch of your repo and watch your Dash app deploy itself to Heroku!

My example repo can be found here on Github. Most of these steps I learned from this testdriven.io course. I am assuming you know what Plotly Dash is and you know how Github works.

A working example of the app can be found on here on Google Cloud. An overview of what we are building:

Example app deployed on Heroku using Github Actions CI/CD Pipeline, image by author

Steps:

1. File Structure
2. Create Plotly Dash App
3. Create a Dockerfile, Run Locally
4. Build Docker Image with Github Actions
5. Create Heroku Account and App
6. Deploy to Heroku through Github Actions

Init

Create a Github repository, preferably, with a README and set it up on your local machine (git-guide).

1. File Structure

We will build the following file structure within a parent folder (docker-dash-example in my case):

├── .github
│   └── workflows
│        └── main.yml
│
├── project
│   ├── app
│   │   ├── __init__.py
│   │   ├── app.py
│   │   ├── functions.py
│   │   └── assets
│   ├── Dockerfile
│   ├── Dockerfile.prod
│   └── requirements.txt
│
├── release.sh
├── setup.cfg
├── .pre-commit-config.yaml
├── .gitignore
│
└── README.md

This structure can be broken down into 4 parts, all explained in more detail later:

1. The .github folder contains our workflow main.yml file that is used in Github Actions for the CI/CD pipeline (step 4 and 6).
2. Project folder contains a folder with the Plotly Dash App, its assets (.css file in our case), the requirements.txt file containing the necessary packages, a Dockerfile for local deployment and a Dockerfile.prod for deployment on Heroku (step 2 and 3).
3. release.sh is used to release the app on Heroku in the final step (step 6).
4. (Optional) .pre-commit-config.yaml and setup.cfg files are used for code styling and linting but can be used for much more (part of step 2).

2. Create Plotly Dash App

Here I will share the app.py file residing within the app folder in the project folder. Don’t let this example app limit your creativity when creating your own app!

Two important things to note here:

1. We put app.server into a server variable.
2. There is nothing like app.run_server to run the app.py file like you would do locally during development (see code block below). This piece of code is excluded because we will run the app with a command from the Dockerfile to run it using a Gunicorn WGSI Server.
3. We import the plot_regression function from functions.py.

if __name__ == "__main__":
    app.run_server(host="0.0.0.0", port=8050, debug=True)

Note that this app component is interchangeable with many Python or non-Python web frameworks.

Pre-Commit Hooks (Optional)

A pre-commit hook can perform multiple checks and identify simple issues before committing code to your repository. Installation is straightforward and many hooks are readily available. The pre-commit hooks used in this repository can be found here, it consists of flake8, black and isort. I use these hooks to keep the code well-formatted (black and isort) and make sure there is no redundant code or simple mistakes (flake8).

3. Create a Dockerfile

Docker is a great way to containerize applications, making them independent of your local machine’s configuration and easily deployable to the various platforms out there. Make sure you have docker installed on your local machine before you continue (download & install Docker).

To run the Plotly Dash app inside a container we create a Dockerfile within our project folder. This Dockerfile looks as follows:

The commands have the following functions:

• FROM: pull the Python image with tag 3.8.6-slim-buster which is a lean version of Python 3.8.6 containing the minimal packages required to run Python, briefly explained here.
• LABEL: optional. Name and E-mail of the maintainer.
• WORKDIR: set the working directory within the container.
• COPY requirements.txt: copy the requirements.txt file to the container.
• RUN pip install — upgrade: upgrade pip to make sure you are running the latest version.
• RUN pip install -r: install all the packages from the requirements.txt file.
• COPY /app: copy the /app folder to the WORKDIR of the container.
• RUN useradd: adding a user to use instead of root (following Heroku guidelines).
• USER: switch to the newly created user.
• CMD: run the Plotly Dash app using a Gunicorn WSGI Server, calling the server variable from the app.py file. Gunicorn handles everything in-between the web server and our Dash App. Note that this Dockerfile can be used for local deployment now.

Run Locally

To run your Plotly Dash app (or my example one) on your local machine with the Dockerfile above you have to build the image and run the container assigning the previously appointed port to it.

Steps to run a Docker Container, from this Dockerfile breakdown

In your terminal go to the parent folder of the project and run the following command to build the image using your Dockerfile.

docker build -t docker-dash project/.

The -t flag is used to tag the container with a proper name, in this case “docker-dash”. After tagging the container we specify the context (path) which contains your Dockerfile and the other files you need for your application.

Now run the image containing your app with the following command:

docker run -p 8050:8050 docker-dash

Here, the -p flag stands for publish which indicates which port of the host machine to bind to the port of your container. Remember that the container is using your local machine as a resource but is running independently. To interact with the Plotly Dash app’s Gunicorn server you need to specify the port that the server is using: 8050 (as denoted in our Dockerfile). Open a browser window and go to https://localhost:8050 to see your app in action, running inside the container.

Now that the app is running locally we will focus on deploying the container with the app to Heroku using a CI/CD pipeline in Github Actions. This pipeline will trigger on a git push to the main branch and looks as follows:

Github Actions pipeline, image by author

4. Build Docker Image with Github Actions

Github Actions is a great way to automate your workflow and opens a world of possibilities. We will utilise Github Packages to store our Docker image and use it later in the deployment step to Heroku.

Personal Access Token

Start by creating a personal access token and selecting these scopes:

• write:packages
• delete:packages
• workflow

Selecting Secret Scopes on Github, image by author

This token can be used in a workflow by calling GITHUB_TOKEN as described here. With this token we can make use of our Github Packages securely.

Workflow: Build

For the first step of the CI/CD pipeline we will create a main.yml within the .github/workflows folders, this structure shows Github this file is a workflow for Github Actions. A .yml file is often used for writing configuration files. Our main.yml file will look as shown below.

The ‘name:’ and ‘on: push: branches:’ should be self-explanatory. ‘env:’ sets the global environment variables for the workflow, in this case the (future) location of our Docker image within Github Packages.

Under jobs we specify one job: ‘build:’, this job contains the following steps:

• ‘name:’ and ‘runs-on:’ specify the name of the job and the runner environment respectively.

1. ‘Checkout master’: Using the checkout@2.3.4 action allows us to checkout the master branch and fetch the latest commit.
2. ‘Log in to Github Packages’: Using the previously created token we log in to Github Packages making sure the GITHUB_TOKEN is available as a local environment variable, if created it will be available in our Github Secrets.
3. ‘Pull image’: We pull the image if it already exists in the global ‘env.IMAGE’ location.
4. ‘Build image’: We build the image using cache from the pulled image to speed up the building process. Note that here I am using the ‘- -file’ flag to point to our Dockerfile. When writing the deployment job we will change this to point to a Dockerfile.prod, which will be explained in the final part of this guide.
5. ‘Push image’: Finally, the updated image is pushed to Github Packages to overwrite (or create) the app package.

This pipeline will run when you push this file to the master branch of your repo. Note that the first time the image does not exist within your Github Packages (the container registry), so it will be created from scratch and pushed to the registry. Once the job is done you can see the newly created image in your Github repo/packages, it should look like this:

Example of the image created with the main.yml pipeline in your Github Packages, image by author

5. Create Heroku Account and App

Heroku is a cloud platform as a service that offers a great free tier for anyone to host a low traffic, low computing power application. With the right infrastructure, it can also easily scale. Your application will be run on one of their free Dynos.

Go to Heroku, create an account and install the Heroku Command Line Interface. Now we will create a named app in the correct region using:

heroku create YOUR_APP_NAME --region YOUR_REGION

Now you can see the (empty) app in your Heroku dashboard here. For your Github workflow to get access to your Heroku account you need to retrieve your Heroku authorization token using:

heroku auth:token

Now we want to be able to use this token as securely as the GITHUB_TOKEN we retrieved from our secrets before in the build step. Go to your Github repo’s settings and locate the secrets tab, click ‘New repository secret’. Add your Heroku token and name it HEROKU_AUTH_TOKEN so we’ll be able to use it in the deployment step.

Have your Heroku authorization token available within the repo’s secrets, image by author

6. Deploy to Heroku through Github Actions

In this final step, we will complete the main.yml file and make sure the app is deployed to Heroku using our pipeline.

Dockerfile.prod

Before adding the deploy step as a job to our workflow we need to make sure the Plotly Dash app can use an available port on Heroku. We’ll utilise Heroku’s PORT variable available within their environment. Make a copy of your Dockerfile and rename it Dockerfile.prod so we know which one is meant for production. Now change the last line where we bind the gunicorn server to this:

CMD gunicorn --bind 0.0.0.0:$PORT app:server

The container will use Heroku’s PORT variable so it will be accessible through the Heroku url. Make sure to also change the reference in the main.yml build job from Dockerfile to Dockerfile.prod. (example Dockerfile.prod)

# In main.yml build job
- name: Build Image 
  run: | 
     docker build \ 
     --cache-from ${{ env.IMAGE }}:latest \
     --tag ${{ env.IMAGE }}:latest \ 
     --file ./project/Dockerfile.prod \ 
     “./project”

Deployment Workflow

Let’s add the deployment job to the workflow main.yml below the build job.

Notice the ‘needs: [build]’ command tells the deploy step to wait until the build step is finished. In the environment of the deploy job we store our Heroku app name and use it to specify the location of the image in the Heroku container registry (similar to specifying the image location in the build stage on Github).

• The first three steps are identical to the build job where we checkout master, login to the Github Packages (Github docker registry) and pull the image. The next steps are going to deploy the container to Heroku:

1. ‘Build Image’: We build the container, notice the different tag we use that refers to the source image.
2. ‘Log in to the Heroku Container Registry’: Using the HEROKU_AUTH_TOKEN you added to your Github Secrets the workflow can log in to the Heroku container registry.
3. ‘Push to the registry’: Push the Image from Github to Heroku’s registry.
4. ‘Set environment variables’: Set the environment variables so they can be accessed in the releash.sh file.
5. ‘release’: Release the container using the release steps as specified by Heroku (chmod +x changes the release.sh file to be executable).

Note that I made some minor changes to the example release.sh structure that Heroku suggests. Adding ‘set -e’ and ‘Authorization: …’ to exit immediately on non-zero status of the curl request and authorize the request to the bearer of the HEROKU_AUTH_TOKEN for security. Here we can use the HEROKU_REGISTRY_IMAGE and HEROKU_AUTH_TOKEN because of setting them in the ‘Set environment variables’.

Push and Sit Back

Push the changes in your code to your repo and navigate to the Actions tab on Github to see your workflow in action! You should see the two steps defined within your main.yml file:

Two step workflow, image by author

Initially, the workflow can take some time installing the packages as it does not have any cache to speed up the building step.

Check out your App!

Go to https://YOUR_APP_NAME.herokuapp.com to see your app in action, also in your Heroku dashboard you can see the app up and running.

Conclusion

Finally, we have Github Actions, Dash, Docker and Heroku working together to deploy your app online for free (not free anymore)! Github Actions and Docker make it possible to deploy an app regardless of the app framework or the cloud provider. If you’d like to deploy to a different cloud platform, make sure to check out these intros for Azure, AWS or GCP.

Hopefully, this article enriches your understanding of the complete pipeline as well as its individual parts.

Thanks for Reading

I hope it was helpful! I have learned a lot from figuring out the details and laying it out as structured, compact and detailed as possible. Please don’t hesitate to ask questions or give feedback.

Part 2 elaborates on how to deploy to Google Cloud Platform!

Happy to connect on LinkedIn!

Some of my other projects.

Things to Improve:

• There can (and should) be a testing step in between the building and the deployment steps of the CI/CD pipeline.
• Using multi-step building can speed up deployment.
• The docker image can be made smaller with the right tweaks, meaning faster building and deployment.

Additional Things to try:

• Deploying to other cloud platforms (also have free tiers/periods) like: Azure, AWS or GCP.
• Using another app framework of choice, like Streamlit, Flask, or not even with Python.
• Deploying a Dash app without containerizing, this article also contains more background information about Gunicorn, Github and Heroku.
• Testdriven.io FastApi Docker course, great course from which I have learned most of the processes described in this guide.
• Using Dash inside a Flask app, explains how Dash can also be a part of a larger Flask app structure.