Tweet Escalation to Your Support Team — Sentiment Analysis with Machine Learning
Solution to automate tweet sentiment processing for airline support request escalation
I will explain end-to-end technical solution which would help to streamline your company support process. I will focus on airline support requests received from Twitter. It could save a lot of time and money for the support department if they would know in advance which request is more critical and must be handled with higher priority.
What will be covered in this article:
- Use case explanation
- Machine Learning solution to predict tweet text sentiment — check support request escalation is required or not
- Accessing Machine Learning model in AWS SageMaker
- Backend implementation with Node.js and Express
- Client UI implementation with JavaScript and Oracle JET toolkit
Diagram explains how all technical bits are connected together.
Let’s start from use case explanation. I thought it would be interesting and useful to implement use case, which would allow reporting tweets sent by people to airline companies where people are asking help. Airlines must be getting lots of such tweets daily, some of them are not relevant at all. My goal was to pick only those tweets which would require escalation and review by a human. This should improve support team results and perhaps help to keep more loyal customers.
I have implemented Machine Learning model for tweet sentiment analysis inspired by instructions given in this book (Chapter 4) — Machine Learning for Business. I highly recommend reading this book for anyone who is looking for practical recipes on how to implement Machine Learning in their company.
Based on the airline (it is not limited to airline only, it can be any Twitter account) account, we fetch 100 latest tweets written to that account. Only those tweets which require escalation and must be handled by human are displayed in the list. Even tweet is classified as the one to be escalated, we can identify by probability score how likely it is to be escalated (see red, yellow and green points). This helps to filter out tweets further, even within the list of tweets picked for escalation.
AmericanAir result — 24% of tweets to be escalated and handled by human:
Delta result — 21% of tweets to be escalated and handled by human:
Emirates result — 19% of tweets to be escalated and handled by human:
British Airways result — 32% of tweets to be escalated and handled by human:
Lufthansa result — 28% of tweets to be escalated and handled by human:
It seems to be a common pattern about 20% — 30% of tweets require escalation for all airlines.
Support person could click on twitter account name or More link to get more details about the tweet and respond accordingly.
I was using AWS SageMaker and BlazingText Machine Learning algorithm to build a model for sentiment analysis. You can read about BlazingText and how to run it in AWS SageMaker in this book (Chapter 4) — Machine Learning for Business, all is described step by step and really simple to understand. You don’t need to purchase the book to download source code (get it from source code link). Read more technical details in my previous post — API for Amazon SageMaker ML Sentiment Analysis.
BlazingText sentiment analysis model is trained and built using dataset uploaded to Kaggle by Stuart Axelbrooke from Thought Vector (www.thoughtvector.io/). The original dataset can be viewed here: www.kaggle.com/thoughtvector/customer-support-on-twitter/home. Along with source code, you can download the processed dataset. Book authors processed original dataset — they removed all of the tweets except the original tweet and used the responses to generate label the original tweet as escalated or not escalated. There are approximately 500 000 tweets used to build support escalation sentiment model.
When Machine Learning (ML) model is built in AWS SageMaker, you need to create an endpoint, which would allow to access model from outside and call predict function. Endpoint creation is included into Jupyter notebook.
With active endpoint available, we can create AWS Lambda function which would act as a proxy between ML model and REST API. Lambda allows to build data transformation logic before calling ML model and process result returned by predict function before the result is sent back to REST API. Read more about it here — Amazon SageMaker Model Endpoint Access from Oracle JET.
Sample code for Lambda function:
ENDPOINT_NAME variable points to customer-support endpoint name created while running Jupyter notebook. Next step is to define REST API in AWS (read more about it in the posts referenced above):
Make sure to enable CORS support for this API. CORS allows to call API from the external host — for example from JavaScript application:
To summarize Machine Learning part:
- You need to run Jupyter notebook, train/validation model. Create AWS endpoint once the model is built
- Define AWS Lambda function to access ML model
- Define AWS REST API to pass incoming requests to AWS Lambda
Next, we move on to backend implementation with Node.js and Express. I have generated Node.js app with express application generator. Read more about available commands here. You get app.js file, which acts as a central router — this is the place where you can define all REST endpoints handled by the backend. Each endpoint can be mapped to a separate module — invoicerisk.js or twitter.js in the below example. This brings more flexibility and allows to structure code into separate modules:
Express routing code is created by Express generator. In app.js we can define a path to the Node.js module. We will be using this path to execute REST API call from UI.
You can run Node.js app generated with Express with the following command:
DEBUG=<appname>:* npm start
Why we are using Node.js backend at all? The reason is pretty obvious — to access Twitter API we need to provide keys mapped to Twitter account, we can’t keep these keys on the client. Once request will come from the client, it will go to Node.js backend POST handler method. In this method, we will fetch latest tweets through Twitter API, prepare the dataset for classification and pass it to AWS SageMaker through REST API call. The response will be parsed to construct result which will be sent back to the client. Check comments inline the code snippet:
The main task of the backend — help to fetch tweets, send them for classification to Machine Learning model and construct response which can be processed by the client.
UI client is created with Oracle JET toolkit and JavaScript. I’m using Oracle JET because it comes with a nice set of out of the box UI components, allowing to build good looking and functional enterprise application.
You can run Oracle JET app from a command line:
ojet serve
JavaScript code which executes backend call and constructs array of tweets with support escalation request.
This code is using a number of Oracle JET libraries. I would recommend to go through Oracle JET cookbook, it provides an interactive list of examples for each UI component.
Chart and list components are rendered by Oracle JET tags:
My goal was to show how Machine Learning model for sentiment analysis fits into enterprise application architecture, including backend and client-side UI implementation. I hope this practical use case example of airline support request escalation would help to apply Machine Learning in your organization and give ideas how to solve your own business challenges.
