The world’s leading publication for data science, AI, and ML professionals.

I’m Making Work Efficient With an “Expert” AI Assistant

Demonstrating the power of language modelling for improved efficiency in the workplace

Hands-on Tutorials, Data Science and Machine Learning in the Wild

Photo by Luis Argaiz on Unsplash
Photo by Luis Argaiz on Unsplash

What do people do at work?

I love my job as a Data Science lead. I’m always exploring areas where artificial intelligence could add value. I see a massive opportunity right now in natural language understanding (NLU).

Let’s explore it by forgetting models for a bit and focussing on people. What do people actually do at work? For many of us workloads are mostly reading, comprehension, and communication. In my experience 90% of non-meeting activities follow this process.

Reading -> comprehension -> communication

In my industry (banking) legal, compliance, and risk departments are saturated with these activities. Banks hire teams of lawyers, compliance officers, and risk managers to interpret lengthy documents and advise activities. The workloads can be huge causing massive inefficiencies.

I spotted an opportunity: how about an AI assistant that could answer basic regulatory queries and help with more nuanced ones?

Is there a language model powerful enough to do this?

The 175 billion parameter model

GPT-3 is the latest of the generative pre-trained models released by [Open AI](https://beta.openai.com/overview). It’s pretty much the state-of-the-art in language modelling right now with 175 billion parameters. This allows it to do funky stuff like sequence-to-sequence modelling, meta-learning, and generative modelling. In plain English it appears to be able to perform most language tasks including complex comprehension. Open AI have released the GPT-3 API to the public and developers are already building some interesting applications.

Some cool ones:

  • SQL Translate
  • Movie to emoji generator
  • Language Translation
  • Python to natural language

Expert Bot proof of concept

I was convinced by GPT-3, but I knew I had to bring my peers at work along with me. I wanted to show them something that would get them inspired about language models, so I decided to build a proof of concept. I’ve called it Expert Bot.

It’s built on top of the GPT-3 API and it’s surprisingly simple. It takes in a knowledge base, has a hardcoded context, and uses the GPT-3 engine to answer questions related to its knowledge base.

Here’s how I built it

  • Open AI account: At the time of writing, they give you $18 of free API calls.
  • Private Keys: Once you’ve signed up, Open AI will give you an organization key and an API key.
  • Anaconda: Need I say more, this made managing the environment and ensuring all the package dependencies were installed easy.
  • Libraries: You’ll need these to get the bot working. openai: Library for the GPT3 API. streamlit: Library for creating web based dashboard/GUIs.
  • A python IDE: I used PyCharm community edition.
  • A knowledge base: This needs to be in .txt format. I grabbed some stuff on GDPR off wikipedia for the purpose of the demo.

Step 1: Set-up

You’ll need to create a PyCharm project and ensure it’s pointed to Anaconda. You can create your own virtual environment but I would definitely recommend using Anaconda.

Installation: You’ll need to install all the libraries with pip. If you’re working in PyCharm, simply open the terminal with alt + f12. Then pip install all the libraries mentioned in the prerequisites.

The Python code

I wrote the bot in 67 lines of code end to end.

Part 1: Import all the libraries.

Part 2: Make the API call to Open AI.

Part 3: Set up the web GUI for the expert bot.

Part 4: Create your knowledge base

The knowledge base is a .txt file that the Expert Bot reads, comprehends, and answers from. You can make the bot an expert in anything by providing an appropriate knowledge base. For my demo, I used online articles and Wikipedia pages to curate knowledge bases on specific topics including GDPR and ESG as these are relevant in my industry right now. I curated the knowledge bases by simply copying the text and pasting it into the .txt file.

The GPT-3 answers API requires the knowledge base to be converted into a jsonl format. I have done this using a python script that reads every line in the .txt file and re-writes it as a dictionary having "text" as the key and the text line as the value. The last line of code uploads your knowledge base file to OpenAI’s file repo.

The code above will create a "knowledge_jsonl.jsonl" file that looks something like this:

Knowledge base in Jsonl format
Knowledge base in Jsonl format

Part 5: Query GPT3 and print answers to UI.

The answers engine itself is simply a framework provided by Open AI that sits on top of the GPT3 model. You need to provide it with a few things to get desirable results.

  • Search_model & model : These are basically the models used for comprehension. GPT3 has different levels which perform well at different tasks. The most "intelligent" is Davinci, which can perform tasks that require comprehension of context. Ada is the least complex, but the fastest for tasks like parsing text and classification. There is a monetary and latency cost associated with each model, the more intelligent the model the more expensive and slow it is.
  • Question: This is the question the user wants to ask the Expert Bot. I have set this up to be passed in from the streamlit UI created in part 3.
  • File: This is simply the file id for your jsonl knowledge base as it is stored in OpenAI’s repo.
  • Examples and examples_context: You need to provide the model with an example of questions to guide it towards the tone and answer format you want. You’ll need to tailor this to your own application.
  • Max_tokens: This is simply the maximum number of tokens for the answer the bot generates. If you just want short answers you should shorten this.
  • Max_rerank: This is the maximum documents to by ranked by search. A document corresponds to each dictionary line in your jsonl file. Higher values improve the accuracy of your results but have a latency cost.

    You can run the application from the terminal in your IDE. Hit alt+f12 and type the following code into your command prompt.

    If you’ve followed the instructions you’ll get something that looks like this…

Expert Bot with GDPR knowledge base which I pulled off Wikipedia
Expert Bot with GDPR knowledge base which I pulled off Wikipedia

Expert Bot in the wild

I’ve had a lot of fun showing this off to colleagues. I can visibly see them get inspired when I show them what it can do. The biggest win is hearing them talk about other use cases for language models!

Without rigorous testing, I can say that the bot’s performance has been adequate for demos using the GDPR and ESG knowledge bases I gave it.

Here are some of the GDPR based questions and answers:

Explain GDPR

🤖 Expert Bot’s Answer> The General Data Protection Regulation (GDPR) is a regulation by which the European Parliament, the Council of the European Union and the European Commission intend to strengthen and unify data protection for all individuals within the European Union (EU).

What are the principles of GDPR?

🤖 Expert Bot’s Answer> Accountability, Privacy by design, Data minimization, Transparency, Lawfulness, and Security by design.

What is privacy by design?

🤖 Expert Bot’s Answer > Privacy by design is a privacy-enhancing approach to the development of information systems. It is a concept that has been around since the 1980s, but has been given new life by the European Union’s General Data Protection Regulation (GDPR).

I have also tried training it on my company’s "About Us" page. I found this didn’t work as well, however I’m still experimenting with parameters and knowledge base curation for this use case. I found that although the answers were great, there was a lot of latency in delivering them about 20 seconds or so.

What I have learned is that the power of a good POC is unrivalled, especially when you are trying to introduce something new.

If a picture is worth a thousand words a good POC is worth 175 billion exactly.

Taking it to the next level

But we can’t get too carried away, there’s a lot that needs to be done before I can claim to have made work more efficient.

  • Validation & Testing: This is paramount in any Machine Learning task. You need to make sure that your model is performing to the standards that you expect. My next steps will be to come up ways to test and validate performance of the bot for specific tasks around compliance and risk management, a completely non-trivial exercise. Some work needs to be done to understand what are the key business metrics for these tasks, then I can set up experiments to test how the bot performs against these.
  • Fine tuning: GPT-3 has been trained on the entire internet, but the literature indicates that fine tuning the model on a subset of data reflecting your use cases could improve performance even more.
  • Scaling: As previously mentioned there is a monetary cost to using these models. At what scale of operation does the bot have a net positive contribution? We can only really understand this after the first two points have been considered.

⭐️ I love helping people by sharing my experiences of data science in the wild. If you’re not already a member, consider subscribing to Medium to receive more helpful content from me.

Join Medium with my referral link – John Ade-Ojo


Related Articles