Learn to Build Machine Learning Services, Prototype Real Applications, and Deploy your Work to Users

In this post I show readers how to expose their machine learning models as RESTful web services, prototype real applications by attaching a front-end to those services, and deploy their applications using containers.

I believe these 3 areas represent critical skills Data Scientists should have in order make their work relevant and impactful.

The 3 tools I will cover in this tutorial are:

1. Jupyter Notebooks (in JupyterLab) with the Kernel Gateway (turning machine learning workflows into web services);
2. Azle (rapid front-end prototyping library);
3. Docker (deploying applications inside isolated, portable containers);

Why should Data Scientists turn their machine learning into web services?

Regularly creating endpoints to machine learning code forces us to think about abstractions. We need to expose our work to other members of our team on an ongoing basis. This is what holds our workflows accountable to product concerns, and invites non-practitioners into the machine learning conversation.

Why should Data Scientists prototype front-end applications?

We don’t do machine learning for Data Scientists we do it for end users of a product. This means we need to remain cognizant of how our models will be consumed, not just created. Connecting our data pipelines to a front-end provides everyone else on the team a view of what machine learning is capable of, and allows our machine learning to benefit from non-technical feedback.

Why should Data Scientists isolate their work inside containers?

Containerization makes applications portable and reproducible. Portability is a challenge for any application, but machine learning software is particularly susceptible to dependency issues. Data Science is made possible thanks to an array of open source libraries pieced together with custom code. Without containers there is little guarantee things will run consistently when we change environments. This helps us collaborate effectively and allows us to ship applications inside and out of organizations more seamlessly.

Let’s get started…

The Challenge: Say I started a company that specializes in flowers. I hire people regularly to take in shipments and label the flowers prior to stocking. While most flowers come labelled from the wholesaler, enough don’t to justify my need to train employees on how to recognize flowers. Training employees is time consuming and expensive.

The Solution: I am going to create an application that helps train my employees quickly. Since studies show competition leads to better learning, my application will allow new hires to compete against an AI at recognizing flowers based on photos and measurements.

The Mockup: Low-fidelity mockups help us explore ideas without restricting ourselves to specific engineering practices or protocols. In this tutorial I will use the Iris dataset to create an application that helps users train to identify Iris flowers, and compete against an “AI” doing the same.

Our application will contain 3 parts. Our first part will allow users to see random Iris flowers along with their measurements. This will be their training area, used to practice relating flower measurements to their label.

Our second part will allow users to challenge our AI, seeing who (or what) can get more correct guesses.

Our third part will show the current scoreboard with real-time scores based on the performance of my trainees and the AI. I will review these reports with new hires to gauge their training.

Building the entire application is beyond the scope of this article. We will focus on building the first section, leaving the remaining code for you to explore on your own.

By the end of this article we will have built the front-end and back-end of the following machine learning application:

Machine Learning Application Built Using Azle and JupyterLab

BACK-END versus FRONT-END

We need both a back-end for processing and a front-end to provide interaction to users. Our back-end will include the machine learning we prototype in a Jupyter notebook, while the front-end will be a web application built using Azle.

Since we are building an application that recognizes Iris flowers it makes sense to write a data pipeline that gathers and validates Iris flower data, trains and validates a model capable of classifying Iris flowers, and deploys that model as a service for front-end consumption. Since rapid prototyping is the goal I will leverage an existing machine learning pipeline to sit inside my Jupyter notebook. Google’s Tensorflow tutorial has an end-to-end example that uses Deep Learning to classify Iris flowers. Since the Iris dataset is small, and my app does not require interpretability, deep learning should work fine.

Install JupyterLab

We will use JupyterLab as our development environment. JupyterLab allows us to work with an IDE inside our browser, which is highly convenient when doing both machine learning work and web application development.

Download and Install JupyterLab

Once installed, create a new folder in Desktop (or any location you prefer), naming it flower_power. Open terminal and change into your new flower_power directory. Once inside, run the following to open JupyterLab:

jupyter lab

This will automatically open JupyterLab in your default browser as a new tab:

In the launcher choose a new Python 3 Notebook. Rename the notebook flower_power.ipynb

If you’re new to machine learning take the time to follow along Tensorflow’s tutorial as you add its code to consecutive cells in your flower_power notebook. It’s a nice overview of common tasks in machine learning. If you already know the drill simply add the code to each cell and click through your notebook to ensure all dependencies are satisfied. Or you can simply download the full Jupyter notebook on GitHub here. Simply use Jupyter’s upload utility to load the file.

Now, our JupyterLab has our flower_power.ipynb notebook ready to use:

The flower_power.ipynb Jupyter Notebook

Learning to Expose Jupyter Cells as REST Endpoints

In addition to enabling data science in the browser Jupyter can also turn our machine learning into a web service. By adding the Jupyter Kernel Gateway we can expose any cell in our notebook as an endpoint. Think of an endpoint as a gateway between our notebook and any application needing to consume its outputs. Creating endpoints means work we do in machine learning can be exposed to applications and their developers.

Install Jupyter Kernel Gateway

In your terminal run the following:

pip install jupyter_kernel_gateway

We will see how to use the gateway shortly. At this point we want to choose cells in our notebook to expose as endpoints. This is as simple as adding the following comment to the beginning of the cell:

# GET /my_path

We will replace “my_path” with the name of our endpoint. First, let’s create a function that returns a random Iris flower from the test dataset in our flower_power.ipynb notebook:

This function chooses a random flower name from 3 possible choices, selects the URL for that random choice, and retrieves the data associated with the flower.

We will call this function using an endpoint so our application can fetch a random flower name, its image URL and the data associated with its 4 measurements.

I added the image URLs to the above function by going to Wikipedia’s Iris Data Set page and simply right-clicking the appropriate image in the Data set section. I selected Copy Image Address and pasted the URLs into the above function as url_1, url_2, and url_3.

We want to turn this function into an endpoint, making it callable from our web application in the browser. As shown above, we use the Kernel Gateway by wrapping the following around our code:

• a GET comment on the first line of the cell;
• a print command that outputs JSON.

Now, when we start the kernel gateway (later) this code will be executed by pointing our browser to the following URL:

localhost:9090/get_random_flower

Don’t worry about the 9090 port yet, I’ll show how to set this later. This endpoint will run the show_random_iris() function created above, and add the results to a dictionary called res (since we are using JSON be sure to add import json to the top cell in your notebook). What we get back is a JSON response that looks like this:

You must add the GET comment to the first line of the Jupyter cell you want as an endpoint. If you place any comment or code above your GET line that cell will fail.

To check our endpoint lets setup the Jupyter kernel gateway. Since we have already installed Jupyter gateway we just need to enter the following command in terminal, inside the same directory containing our flower_power.ipynb notebook:

jupyter kernelgateway --KernelGatewayApp.api='kernel_gateway.notebook_http' --KernelGatewayApp.ip=0.0.0.0 --KernelGatewayApp.port=9090 --KernelGatewayApp.seed_uri=flower_power.ipynb --KernelGatewayApp.allow_origin='*'

This command does the following :

• starts Jupyter gateway;
• sets the base URL to localhost (0.0.0.0);
• exposes port 9090 to browsers (you can change this to any available port);
• assigns the flower_power.ipynb notebook to the gateway;
• allows incoming traffic from everywhere (remember, we’re just prototyping)

Running the command will produce outputs related to Tensorflow (ignore), and our registered resources (endpoints). The last line should read:

Jupyter Kernel Gateway at http://0.0.0.0:9090

Open a new tab in your browser and point to:

localhost:9090/get_random_flower

You should see the data returned. Keep hitting refresh to see the output in your terminal return a different flower each time. We are now calling python code, sitting inside a Jupyter notebook, all through our browser. You have now built a service. Since our machine learning will be written in Python creating a machine learning service is no different. We simply choose the functions we wish to call via our browser.

To check our endpoint, we should use common tools developers use to do the same. One such tool is Swagger. Swagger is software that helps developers design, build, document, and consume RESTful Web services. It’s a common way to pass off service endpoints to developers on a team. A great feature of Jupyter’s Kernel Gateway is it automatically generates a swagger.json endpoint we can access using SwaggerUI.

This is what allows developers to “mashup” applications using different API endpoints. As Data Scientists, we want our models to be part of that mashup so our organization can see the value machine learning brings their products. Developers within your organization will likely have their own Swagger UI installed (or something similar) so you can simply hand-off your swagger endpoints to them.

For our own checking, let’s download Swagger UI from Dockerhub and run the UI application, checking the endpoint we just created with the Kernel Gateway. If you do not have Docker installed, Download and Install Docker first. With Docker installed and running, open a new tab in terminal and run the Docker pull command to download DockerUI:

docker pull swaggerapi/swagger-ui

Type docker images in your console to see the following:

This shows the name of our new image, swaggerapi/swagger-ui. We can now use that name to run its container:

docker run -p 80:8080 swaggerapi/swagger-ui

… and open swagger-ui in our browser on port 80.

In a new tab in your browser navigate to the following URL :

http://localhost:80

At the top of the screen is the explore bar, where we add the path to our swagger.json file; the one exposed by our Jupyter Kernel Gateway. Add the following to this bar (make sure http:// is included in the URL):

http://localhost:9090/_api/spec/swagger.json

Since we are simply hitting our existing endpoint in the service we created the port is still 9090. Hit enter and see what happens. You should see the following appear on the screen:

This is our GET endpoint exposed inside Swagger. Click on the GET button, then on Try it out, then on Execute:

This information is useful to your developers. They can see how to access our API via the curl command in terminal, see the full Request URL, see the response body, and any headers associated with that response. We can also see the success code 200 showing our endpoint worked as expected.

Since we did not set any parameters on our endpoint there is nothing to query. Normally, developers would run queries to decide how to best bring our machine learning into the application. We will show how to set and pass parameters later when we build our Flower Power application.

We now have a sense of how Data Scientists can create web services for any code inside their Jupyter notebooks. While we are only using a simple example here, this approach exposes all kinds of machine learning functionality to stakeholders inside and outside our team.

We are now ready to start prototyping a real application. While normally within the purview of developers, there are strong reasons for Data Scientists to do this themselves. Real applications give our machine learning “legs”, and remove much of the ambiguity around how machine learning will contribute to an organization’s products.

Of course Data Scientists have enough on their plate without diving deep into HTML, CSS, and JavaScript, let alone custom frameworks for engineering large-scale web applications. What we need is a rapid front-end prototyping library, based on intuitive syntax, that connects to API endpoints with ease.

Some machine learning practitioners will argue their specialization precludes the time/need to learn front-end development; I have found this usually stems from apprehension around learning how to develop software rather than a genuine belief they don’t need to. It’s important to realize that prototyping software and developing production software are different art forms. With the right high-level library, prototyping software can be extremely rewarding to domain experts. There is nothing more empowering to a professional in today’s information-driven economy than an ability to quickly convert their vision into a real-world tool people can use.

Learning to Prototype Applications with Azle

Too often Data Scientists work on improving their machine learning models in isolation, only tossing results over the fence once satisfied with their models. This increases the risk of poor ROI on projects since machine learning in a vacuum never benefits from real-world feedback. An application’s front-end is the only touchpoint real users have to the outputs produced by learning algorithms. Data Scientists need to inform their pipelines based on how people touch and feel a product.

Azle is a front-end prototyping library designed specifically for rapidly crafting applications. It looks to strike the right level of abstraction between flexibility and ease-of-use. It uses functions to add and style content, bind events, and interact with API endpoints, making it easier to build full-powered, data-driven applications.

All web applications begin with an html template that will eventually hold the elements that make the front-end of an application possible. Our first step is to use the template provided by Azle. You can copy and paste the following into a file called index.html:

HTML template for Azle.

All our Azle functions can be added between the <script> tags at the bottom. In larger applications you would keep your scripts in separate folders and load them from this index file using Azle’s load_script function. We will place most of our Azle functions between script tags at the bottom for brevity, but we will load one additional file later on.

Azle is an intuitive language, where we write code based on what we are doing. Azle comes pre-baked with minimalist styling so we don’t have to spend as much time getting our application to look modern. It also uses layouts making it easy to arrange content on the screen.

Let’s look at some common functions in Azle. Click on any of the fiddles to play with the actual code:

ADDING CONTENT

az.add_button(target_class, target_instance, {
    "this_class" : "my_button",
    "text" : "CLICK ME"
})

Fiddle

az.add_dropdown(target_class, target_instance, {
    "this_class" : "my_dropdown",
    "options" : ["option A", "option B", "option C", "option D"],
    "title" : "choose option..."
})

Fiddle

az.add_slider("target_class", 1, {
    "this_class" : "my_slider",
    "default_value" : 4,
    "min_value" : 0,
    "max_value" : 10
})

Fiddle

Notice each function uses a target_class and target_instance as the first 2 parameters. This tells Azle where to place the content, or which content we wish to style. For example, in the slider above, we first create a section to hold our application, then add the slider to that first section:

az.add_slider("my_sections", 1, {
}) 

The 3rd parameter is an object containing properties we wish to pass. In our first slider example we passed in a class name (“this_class”) along with default values. Each content and styling has its expected properties, which can be found in Azle’s documentation.

STYLING CONTENT

We can also style any content we add. Styling is done by using target_class and target instance as above, and passing style properties into a style object as the 3rd parameter. For example, styling our button above:

az.style_button('my_button', 1, {
    "background" : "hotpink",
    "border-radius" : "6px",
    "width" : "200px",
    "color" : "black",
    "cursor" : "wait"
})

Fiddle

Note that Azle styles follow the standard CSS approach to styling documents.

ADDING EVENTS

We can bind events to our UI elements. Events are things like clicking, hovering and selecting. Here we add a click event to our button, using Azle’s add_event function:

az.add_event("my_button", 1, {
    "type" : "click",
    "function" : "alert('you clicked me')"
})

Fiddle

For a look at all available events in Azle see here.

GRABBING VALUES

Often we want to grab a value from an input or dropdown, before calling an event. This is accomplished using Azle’s grab_value() function:

az.grab_value("my_dropdown", 1)

For example, we could add a button whose text comes from a selection made in a dropdown menu:

az.add_button("my_sections", 1, {
    "this_class" : "my_button",
    "text" : az.grab_value("my_dropdown", 1)
})

Fiddle

CALLING ENDPOINTS

Calling endpoints is how we send and fetch data from REST APIs. Azle’s call_api() function asks for the base url, any parameters we need to pass, and the function to call once the response has been received:

Fiddle

ADDING VISUALS

Azle works great with D3.js, as I discuss here. You can also check out the azle_d3 example application.

Here is the preview video showing how Azle can be used to bind D3 visuals to UI elements in your application:

Our brief tour of Azle has shown us the pieces we need to create fuller applications including front-end content, styling, events, and visuals, as well as the ability to call endpoints. Be sure to check out the documentation, which includes a getting started tutorial.

Let’s get started creating our application.

CREATING FLOWER POWER

Creating the entire application mocked up at the beginning is beyond the scope of this article. We will just build the first piece of our application, then look at what to consider regarding other pieces. The GitHub project contains the complete code.

Here is the video we saw at the beginning, showing the full application:

To begin prototyping our application we start as always, with the html template shown previously, adding functions between the <script> tags at the bottom.

The quickest way to start would be to simply save the html template to your desktop and right-click on the file, choosing Open With, and selecting your browser, as shown below:

However, it is more convenient to have all our files located inside our IDE, which for us is JupyterLab. This way we can open terminals as needed, install libraries, make changes to our flower_power.ipynb, restart our machine learning service, and prototype our front-end; all on a single browser screen. When building real-world software this is highly beneficial. We will use this approach.

If you’ve been following along since the beginning you already have JupyterLab up-and-running inside a folder named flower power, and also have the flower_power.ipynb tab open as your first JupyterLab file.

Let’s get our other tabs open so we have a full prototyping environment. Let’s open a terminal by clicking the New Launcher icon and selecting Terminal beneath the Other option:

Do this twice so we have 2 terminals open. Let’s also create an index.html file for our Azle application. Click on Text File under Other …

… and rename the file index.html. Open this file as the 4th tab in your JupyterLab environment.

Let’s also create 2 folders in our main directory, one called d3_visuals and the other scripts. We create folders in JupyterLab by clicking on the New Folder button:

… then right-clicking the newly created folder and renaming it. You should now have a directory structure that looks as follows:

flower_power
├── scripts
├── d3_visuals
├── flower_power.ipynb
├── index.html

You should also have 4 tabs open for your prototyping environment:

A prototyping environment for building machine learning applications.

We will add 2 additional files later to our d3_visuals and scripts folders, but for now let’s get started building our application.

We will need to view our application in another browser tab, meaning we need to “spin up a web server.” We can do this easily by opening our first terminal tab in JupyterLab and running the following:

python3 -m http.server

This will then show…

Serving HTTP on 0.0.0.0 port 8000 …

Since we ran this command in the same directory as our index.html file, it means we can view our application by opening a new browser tab and navigating to http://localhost:8000/.

Note: See BONUS SECTION at the end for using a different port than 8000.

Of course we haven’t added anything to our index.html file yet so there is nothing to show. Let’s add some Azle functions.

We start with basic styling for the body of our application. We will add a minimalist font (anything from Google Fonts), a “scroll to top” button, and set the widths of our application:

Azle apps also require sections. Sections are just as they sound; different sections of the application that show a specific context. Let’s create 4 sections for Flower Power; 1 for a top navigation/title bar and the other 3 for the 3 mockup designs we created at the beginning.

Since there are sections in our application we now have something to look at, so we can open http://localhost:8000/ in a new browser tab to see our application .We currently have a 4 section application that looks like this:

Let’s set the heights of our sections to conform to our mockups. We will set the first section’s height to be shorter, so that it acts as our navigation/title bar:

The top of Flower Power now looks like this:

In our navigation bar we will want a logo, a title, and a few buttons that allow users to scroll to a specific section. Whenever we want to arrange content on the screen we use Azle’s layouts. Let’s add and style a layout inside the first section, then add an image, text and buttons:

I will not detail every Azle function in this article, however, given Azle’s highly intuitive syntax you should be able to follow which function is doing what. Remember, the more you use Azle the quicker you’ll develop an intuition around how to piece together applications quickly.

Let’s also add our buttons to the 3rd cell in our layout:

While the above may look a little intimidating its quite straightforward. We first store our button titles inside Azle’s hold_value object. We then use Azle’s call_multiple function to iterate over a set of functions 3 times. The functions inside the back-ticks (called a “template string” in JavaScript) are where we add a button, style a button, and add a scrolling event.

Let’s align the 3rd cell contents to the right :

Finally, whenever we are done adding content to our layout we should set the layout’s border to 0 to remove the border. Go back to the style_layout function used previously on the “banner_layout” target class and remove the border:

Our top banner now looks like this:

For the remaining sections I will not step through all the code, rather I will simply point out important Azle functions as they relate to calling our machine learning service. You can see the GitHub project for the complete source code. You can copy and paste the necessary code into your index.html file, or simply clone the entire project and follow along that way.

To build section 2 we add and style a layout, add text, a button, an image, and a visualization. Section 2 looks like this:

NOTE: Our application sections do not have to follow the mockups precisely (which is why they are called low-fidelity mockups). The beauty of rapid prototyping is we can pivot designs quickly to suit changing requirements.

If you refreshed your index.html page you would not see any flower or visualization. For this to work we need to call the first endpoint we created previously, since that endpoint returned the Iris URL and its data. We also need to add a new script to the scripts folder, and a D3 visual to the d3_visuals folder.

If you cloned the flower_power project from GitHub you already have all necessary folders and files. If not, look at the project files on GitHub and add the necessary folders and files to your JupyterLab project. Your complete directory structure should look like this:

flower_power
├── scripts
    └── wrapped_functions.js
├── d3_visuals
    └── barchart.html
├── flower_power.ipynb
├── index.html

Now we just need to get our REST service up-and-running. In your second terminal tab, run the command we looked at previously for starting the kernel gateway:

jupyter kernelgateway --KernelGatewayApp.api='kernel_gateway.notebook_http' --KernelGatewayApp.ip=0.0.0.0 --KernelGatewayApp.port=9090 --KernelGatewayApp.seed_uri=flower_power.ipynb --KernelGatewayApp.allow_origin='*'

Now hit refresh on your index.html page. The first section should return both a random iris image along with its data visualized as an angled bar chart.

At this point your prototyping environment inside JupyterLab is complete and running. You have:

• Terminal 1: Running a web server to host index.html
• Terminal 2: Running Jupyter Kernel Gateway to expose notebook cells as REST endpoints
• flower_power.ipynb: A Jupyter notebook housing your Python or R code
• index.html: A web application built in Azle

You can use this same setup to turn your machine learning into real software prototypes. Since most data scientists and other machine learning practitioners use the Jupyter environment, this setup enables you to quickly attach full-featured front-ends to your Python and/or R code.

An important functionality in section 2 is the ability to call our REST endpoint when the user clicks the SHOW IRIS button. We already know how to add events since we looked at this previously when reviewing Azle functions. We add the button, then attach a click event:

At this point we need to add the function that gets called when the user clicks the button (the above just shows an empty string). Previously we looked at Azle’s call_api() function. We use this to call any endpoints we create. You will find this function inside the scripts folder in wrapped_functions.js.

We use wrapped functions in Azle anytime we want to group many functions into a single named function. For example, if we wanted to create a wrapped function called “fetch_random_iris” it would look like this:

This way, we can group multiple functions between the curly braces above and invoke them by calling fetch_random_iris().

First, let’s look at the group of functions we want to call when the user clicks our SHOW IRIS button, then we’ll add these inside our wrapped function.

The following group of functions do 3 things:

• calls our first endpoint currently exposed by the Jupyter Kernel Gateway;
• calls a “call_function” when the data is returned;
• spins our button after the user clicks.

Let’s add these to our wrapped function called fetch_random_iris:

Notice we added a “call_function” parameter to our fetch_random_iris wrapped function. You can pass any number and type of arguments you want into your wrapped functions. Here, we pass a “callback”, which is a function that gets called once the API has returned its data.

In Azle, we call wrapped functions like this:

az.call_wrapped_function.fetch_random_iris()

So, we can add this to our click event above, such that when a user clicks the SHOW IRIS button it will run fetch_random_iris().

Notice we still need to add our callback function as an argument in the above fetch_random_iris call. We also have a D3 visual as our bar chart, which gets drawn once the data is returned. This is handled by the callback function, which is another wrapped function. As stated previously, I will not go through all the code in this article. I encourage you to look through the GitHub project to see the complete code.

I also mentioned Azle and D3 above. Have a look at my article on how to add D3 visuals to your applications using Azle.

The remainder of the application follows the same approach. We review the low-fidelity mockup, add layouts and UI elements, bind events as needed, group functions as wrapped functions, and use the call_api function to send and receive data from our machine learning service.

Here is section 3:

And section 4:

As we saw at the beginning, here is the final result:

With a real application in hand, it’s time to ship!

Learning to Dockerize your Machine Learning Applications

When we “ship” an application we package everything up and place it onto a server. This allows other people to access the application, rather than having it sit only on our local machine. Getting an application onto a remote server comes with its own technical challenges. Just because our application runs nicely on our laptop doesn’t mean it will do so elsewhere. People use different runtime environments, different versions of libraries, different versions of programming languages, etc.

To make this much easier Docker turned a technology called containerization into an easy to use product. Docker allows us to take a snapshot of our local environment and ship its exact configuration onto any server. This means we can use Docker to containerize the different things our Flower Power application depends on:

• Python 3
• Libraries such as Pandas and Numpy
• TensorFlow
• Jupyter Kernel Gateway

Readers who decided to use SwaggerUI as described previously in this article have already been exposed to Docker. If you haven’t yet, Download and Install Docker prior to taking the next steps.

With Docker installed we are ready to write a Dockerfile. A Dockerfile is a file that tells Docker how to build an “image.” Images specify how containers are to be built. An instance of an image is a container. A key concept in Docker is what are known as base images. Base images allow us to start our container using languages and libraries that have already been packaged for us. In other words, we don’t need to explicitly list all our dependencies in our docerkfile if they are covered in our base image. We will use the jupyter/datascience-notebook as our base image, since many of the machine learning libraries are contained in this image.

Let’s write our Dockerfile:

You can refer to Docker’s tutorial for specifics on why Dockerfiles look the way they do. Here we will just point out a few important points:

• FROM allows us to bring in our base image;
• RUN allows us to install extra dependencies;
• CMD is where we can run a terminal command. Here, we just run the same command we ran previously for starting the kernel gateway.

With our Dockerfile in hand, we can now build our image:

docker build -t flower_power .

We are simply running the docker build command and naming our image flower power.

We make sure our image has been built by running the following command:

docker images

which should show this in the terminal:

Then we run the container:

docker run -p 9090:8888 flower_power

Here we are exposing port 9090 to the outside world. While the terminal will spit out a message saying “Jupyter Kernel Gateway at http://0.0.0.0:8888” this is coming from inside the container. Our access is in fact via port 9090.

We can ensure our image is running by typing the following in terminal:

docker ps

We now have a running container. Deployment can be as simple as transferring our Dockerfile to another server and running the build and run commands there. Docker will pull all the necessary dependencies and expose our machine learning through our specified endpoints. Our front-end application can either be on the same server we deployed our machine learning to, or another server altogher. This is what it means to create a machine learning service that can be deployed onto servers and consumed by other applications. Think of the possibilities.

Setting up a remote server is straightforward. You can use a service like DigitalOcean to create a droplet, then transfer your Dockerfile to that droplet using a transfer protocol (e.g. scp or sftp).

One thing to note is we will need to change the URL used by Azle’s call_api() function, since it will be calling our deployed machine learning service on a new IP address, rather than localhost.

Of course deployment can get much more involved than this, but the basic ideas are the same. Whether it’s continuous integration and deployment, integration testing, microservices, or any other related concept, it all comes down to pushing packaged code onto servers and consuming endpoints.

And that’s it! In this article we have setup a Jupyter notebook with a machine learning pipeline, exposed endpoints using Jupyter’s Kernel Gateway, reviewed those endpoints using SwaggerUI, created a front-end application using Azle to consume our endpoints, and packaged our machine learning into a container that can be deployed to a server. This is a more realistic look at how data science happens on real-world projects.

So go build and deploy your machine learning services, and create interesting applications to consume them. Such skills make for well-rounded Data Scientists who know how to bring their work to life.

P.S.

If you are having trouble on any of these steps, please ask questions in the comments section.

BONUS NOTES

Using Native Python/R Visualizations with Azle

Data Scientists often rely on visuals produced by either Python’s Matplotlib or R’s ggplot2. Azle can consume these visuals when they are exposed as data URIs. Data URIs are how images are embedded in web applications, rather than pointing to a source URL. For example, here is how we normally produce a plot with matplotlib:

import numpy as np
import matplotlib.pyplot as plt
t = np.arange(0., 5., 0.2)
plt.plot(t, t, 'r--', t, t**2, 'bs', t, t**3, 'g^')
plt.savefig('lines.png')

which produces this:

For Azle to consume this plot we simply convert our matplotlib plot to a data URI and expose the serialized output using the kernel gateway.

Here is an example where we use Azle’s call_api function to fetch our encoded plot, then use the add_image function to draw the plot:

For this to work inside a Docker container, we need to create a data folder by mounting a volume. We can do this by creating the data directory inside our main folder, and adding this line to our Docker run command:

sudo docker run -p 9191:9191 -it -v /path_to_dockerfile/data:/data testing

The same approach can be used in R. Here is how we encode a figure in R using the knitr library:

library(knitr)
file_name = 'lines.png'
uri=image_uri(file_name)
cat(uri)

Building More Complete Applications

We only showed the basic way for a Notebook to respond to our requests from Azle using GET. Depending on how sophisticated your application is, Jupyter’s Kernel Gateway supports all CRUD operations. See here for more details on how to more fully define your web api.

Constructing URLs

There are 2 ways one can construct a URL for making REST API calls:

1. As part of the URL-path (i.e. /api/resource/parametervalue )
2. As a query argument (i.e. /api/resource?parameter=value )

From Wikipedia: Unlike SOAP-based web services, there is no “official” standard for RESTful web APIs.[14] This is because REST is an architectural style, unlike SOAP, which is a protocol.

Avoiding Conflicting Ports

• If you are running a Jupyter Notebook at the same time as the kernel Gateway you can specify your notebook port to avoid conflict (or just set your kernel gateway port to something other than Jupyter’s default 8888). For example, you can set any notebook to port 8889 using the following command:

jupyter notebook --port=8889

Making Endpoints in R

If using the R kernel in Jupyter instead of Python, you can handle requests as follows:

# GET /my_path
req <- fromJSON(REQUEST)
args <- req$args
parameter_name = args$parameter_name
# do something with parameter_name
print(toJSON(parameter_name))

Be sure to add the jsonlite library to the top of your notebook.

library(jsonlite)

Happy Building!

If you enjoyed this article you might also enjoy:

Graduating from Toy Visuals to Real Applications with D3.js

GUI-fying the Machine Learning Workflow: Towards Rapid Discovery of Viable Pipelines

Step-by-Step Guide to Creating R and Python Libraries (in JupyterLab)