The world’s leading publication for data science, AI, and ML professionals.

Getting Started with LangChain: A Beginner’s Guide to Building LLM-Powered Applications

A LangChain tutorial to build anything with large language models in Python

"What did the stochastic parrot say to the other?" (Image drawn by the author)
"What did the stochastic parrot say to the other?" (Image drawn by the author)

Since the release of ChatGPT, large language models (LLMs) have gained a lot of popularity. Although you probably don’t have enough money and computational resources to train an LLM from scratch in your basement, you can still use pre-trained LLMs to build something cool, such as:

LLMs are changing how we build AI-powered products

With their weird APIs and prompt engineering, LLMs are changing how we build AI-powered products. That’s why new developer tools are emerging everywhere under the term "LLMOps".

One of these new tools is LangChain.

GitHub – hwchase17/langchain: ⚡ Building applications with LLMs through composability ⚡

What is LangChain?

LangChain is a framework built to help you build LLM-powered applications more easily by providing you with the following:

  • a generic interface to a variety of different foundation models (see Models),
  • a framework to help you manage your prompts (see Prompts), and
  • a central interface to long-term memory (see Memory), external data (see Indexes), other LLMs (see Chains), and other agents for tasks an LLM is not able to handle (e.g., calculations or search) (see Agents).

It is an open-source project (GitHub repository) created by Harrison Chase.

Because LangChain has a lot of different functionalities, it may be challenging to understand what it does at first. That’s why we will go over the (currently) six key modules of LangChain in this article to give you a better understanding of its capabilities.

Prerequisites

To follow along in this tutorial, you will need to have the langchain Python package installed and all relevant API keys ready to use.

Installing LangChain

Before installing the langchain package, ensure you have a Python version of ≥ 3.8.1 and <4.0.

To install the langchain Python package, you can pip install it.

pip install langchain

In this tutorial, we are using version 0.0.147. The GitHub repository is very active; thus, ensure you have a current version.

Once you are all setup, import the langchain Python package.

import langchain

API keys

Building an application with LLMs requires API keys for some services you want to use, and some APIs have associated costs.

LLM provider (required) – You will first need an API key for the LLM provider you want to use. We are currently experiencing "AI’s Linux moment" where developers must choose between proprietary or open-source foundation models based on a trade-off mainly between performance and cost.

LLM Providers: Proprietary and open-source foundation models (Image by the author, inspired by Fiddler.ai, first published on W&B's blog)
LLM Providers: Proprietary and open-source foundation models (Image by the author, inspired by Fiddler.ai, first published on W&B’s blog)

Proprietary models are closed-source foundation models owned by companies with large expert teams and big AI budgets. They usually are larger than open-source models and thus have better performance, but they also have expensive APIs. Examples of proprietary model providers are OpenAI, co:here, AI21 Labs, or Anthropic.

Most available LangChain tutorials use OpenAI but note that the OpenAI API (is not expensive for experimentation but it) is not free. To obtain an OpenAI API Key, you need an OpenAI account and then "Create new secret key" under API keys.

import os
os.environ["OPENAI_API_KEY"] = ... # insert your API_TOKEN here

Open-source models are usually smaller models with lower capabilities than proprietary models, but they are more cost-effective than proprietary ones. Examples of open-source models are:

Many open-source models are organized and hosted on Hugging Face as a community hub. To obtain a Hugging Face API Key, you need a Hugging Face account and create a "New token" under Access Tokens.

import os

os.environ["HUGGINGFACEHUB_API_TOKEN"] = ... # insert your API_TOKEN here

You can use Hugging Face for free for open-source LLMs, but you will be limited to smaller LLMs with less performance.


A personal note – Let’s be honest here for a second: Of course, you can experiment with open-source foundation models here. I tried to make this tutorial only with open-source models hosted on Hugging Face available with a regular account (google/flan-t5-xl and sentence-transformers/all-MiniLM-L6-v2). It works for most examples, but it is also a pain to get some examples to work. Finally, I pulled the trigger and set up a paid account for OpenAI as most examples for LangChain seem to be optimized for OpenAI’s API. Overall running a few experiments for this tutorial cost me about $1.


Vector Database (optional) – If you want to use a specific vector database such as Pinecone, Weaviate, or Milvus, you need to register with them to obtain an API key and check their pricing. In this tutorial, we are using Faiss, which requires no sign-up.

Tools (optional) – Depending on the tools you want the LLM to interact with such as OpenWeatherMap or SerpAPI, you may need to register with them to obtain an API key and check their pricing. In this tutorial, we only use tools requiring no API key.

What can you do with LangChain?

The package provides a generic interface to many foundation models, enables prompt management, and acts as a central interface to other components like prompt templates, other LLMs, external data, and other tools via agents.

At the time of writing, LangChain (version 0.0.147) covers six modules:

The code examples in the following sections are copied and modified from the LangChain documentation.

Models: Choosing from different LLMs and embedding models

Currently, many different LLMs are emerging. LangChain offers integrations to a wide range of models and a streamlined interface to all of them.

LangChain differentiates between three types of models that differ in their inputs and outputs:

  • LLMs take a string as an input (prompt) and output a string (completion).
# Proprietary LLM from e.g. OpenAI
# pip install openai
from langchain.llms import OpenAI
llm = OpenAI(model_name="text-davinci-003")

# Alternatively, open-source LLM hosted on Hugging Face
# pip install huggingface_hub
from langchain import HuggingFaceHub
llm = HuggingFaceHub(repo_id = "google/flan-t5-xl")

# The LLM takes a prompt as an input and outputs a completion
prompt = "Alice has a parrot. What animal is Alice's pet?"
completion = llm(prompt)
LLM models (Image by the author)
LLM models (Image by the author)
  • Chat models are similar to LLMs. They take a list of chat messages as input and return a chat message.
  • Text embedding models take text input and return a list of floats (embeddings), which are the numerical representation of the input text. Embeddings help extract information from a text. This information can then be later used, e.g., for calculating similarities between texts (e.g., movie summaries).
# Proprietary text embedding model from e.g. OpenAI
# pip install tiktoken
from langchain.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()

# Alternatively, open-source text embedding model hosted on Hugging Face
# pip install sentence_transformers
from langchain.embeddings import HuggingFaceEmbeddings
embeddings = HuggingFaceEmbeddings(model_name = "sentence-transformers/all-MiniLM-L6-v2")

# The embeddings model takes a text as an input and outputs a list of floats
text = "Alice has a parrot. What animal is Alice's pet?"
text_embedding = embeddings.embed_query(text)
Text embedding models (Image by the author)
Text embedding models (Image by the author)

Prompts: Managing LLM inputs

LLMs have weird APIs. Although inputting prompts to LLMs in natural language should feel intuitive, it takes quite a bit of tweaking of the prompt until you get the desired output from an LLM. This process is called prompt engineering.

Once you have a good prompt, you may want to use it as a template for other purposes. Thus, LangChain provides you with so-called PromptTemplates, which help you construct prompts from multiple components.

from langchain import PromptTemplate

template = "What is a good name for a company that makes {product}?"

prompt = PromptTemplate(
    input_variables=["product"],
    template=template,
)

prompt.format(product="colorful socks")

The above prompt can be viewed as a zero-shot problem setting, where you hope the LLM was trained on enough relevant data to provide a satisfactory response.

Another trick to improve the LLM’s output is to add a few examples in the prompt and make it a few-shot problem setting.

from langchain import PromptTemplate, FewShotPromptTemplate

examples = [
    {"word": "happy", "antonym": "sad"},
    {"word": "tall", "antonym": "short"},
]

example_template = """
Word: {word}
Antonym: {antonym}n
"""

example_prompt = PromptTemplate(
    input_variables=["word", "antonym"],
    template=example_template,
)

few_shot_prompt = FewShotPromptTemplate(
    examples=examples,
    example_prompt=example_prompt,
    prefix="Give the antonym of every input",
    suffix="Word: {input}nAntonym:",
    input_variables=["input"],
    example_separator="n",
)

few_shot_prompt.format(input="big")

The above code will generate a prompt template and compose the following prompt based on the provided examples and input:

Give the antonym of every input

Word: happy
Antonym: sad

Word: tall
Antonym: short

Word: big
Antonym:

Chains: Combining LLMs with other components

Chaining in LangChain simply describes the process of combining LLMs with other components to create an application. Some examples are:

  • Combining LLMs with prompt templates (see this section)
  • Combining multiple LLMs sequentially by taking the first LLM’s output as the input for the second LLM (see this section)
  • Combining LLMs with external data, e.g., for question answering (see Indexes)
  • Combining LLMs with long-term memory, e.g., for chat history (see Memory)

In the previous section, we created a prompt template. When we want to use it with our LLM, we can use an LLMChain as follows:

from langchain.chains import LLMChain

chain = LLMChain(llm = llm, 
                  prompt = prompt)

# Run the chain only specifying the input variable.
chain.run("colorful socks")

If we want to use the output of this first LLM as the input for a second LLM, we can use a SimpleSequentialChain:

from langchain.chains import LLMChain, SimpleSequentialChain

# Define the first chain as in the previous code example
# ...

# Create a second chain with a prompt template and an LLM
second_prompt = PromptTemplate(
    input_variables=["company_name"],
    template="Write a catchphrase for the following company: {company_name}",
)

chain_two = LLMChain(llm=llm, prompt=second_prompt)

# Combine the first and the second chain 
overall_chain = SimpleSequentialChain(chains=[chain, chain_two], verbose=True)

# Run the chain specifying only the input variable for the first chain.
catchphrase = overall_chain.run("colorful socks")
Output of a SimpleSequentialChain using PromptTemplates and LLMs in LangChain (Screenshot by the author)
Output of a SimpleSequentialChain using PromptTemplates and LLMs in LangChain (Screenshot by the author)

Indexes: Accessing external data

One limitation of LLMs is their lack of contextual information (e.g., access to some specific documents or emails). You can combat this by giving LLMs access to the specific external data.

For this, you first need to load the external data with a document loader. LangChain provides a variety of loaders for different types of documents ranging from PDFs and emails to websites and YouTube videos.

Let’s load some external data from a YouTube video. You can refer to the official documentation if you want to load a large text document and split it with a Text Splitter.

# pip install youtube-transcript-api
# pip install pytube

from langchain.document_loaders import YoutubeLoader

loader = YoutubeLoader.from_youtube_url("https://www.youtube.com/watch?v=dQw4w9WgXcQ")

documents = loader.load()

Now that you have your external data ready to go as documents , you can index it with a text embedding model (see Models) in a vector database – a VectorStore. Popular vector databases include Pinecone, Weaviate, and Milvus. In this article, we are using Faiss because it doesn’t require an API key.

# pip install faiss-cpu
from langchain.vectorstores import FAISS

# create the vectorestore to use as the index
db = FAISS.from_documents(documents, embeddings)

Your document (in this case, a video) is now stored as embeddings in a vector store.

Now you can do a variety of things with this external data. Let’s use it for a question-answering task with an information retriever:

from langchain.chains import RetrievalQA

retriever = db.as_retriever()

qa = RetrievalQA.from_chain_type(
    llm=llm, 
    chain_type="stuff", 
    retriever=retriever, 
    return_source_documents=True)

query = "What am I never going to do?"
result = qa({"query": query})

print(result['result'])
Output of the RetrievalQA (Screenshot by the author)
Output of the RetrievalQA (Screenshot by the author)

Wait a minute – Did you just get rickrolled? Yes, you did.

Memory: Remembering previous conversations

For applications like chatbots, it is essential that they can remember previous conversations. But by default, LLMs don’t have any long-term memory unless you input the chat history.

Chat with and without conversational memory (Image by the author made with ifaketextmessage.com, inspired by Pinecone)
Chat with and without conversational memory (Image by the author made with ifaketextmessage.com, inspired by Pinecone)

LangChain solves this problem by providing several different options for dealing with chat history :

  • keep all conversations,
  • keep the latest k conversations,
  • summarize the conversation.

In this example, we will use a ConversationChain to give this application conversational memory.

from langchain import ConversationChain

conversation = ConversationChain(llm=llm, verbose=True)

conversation.predict(input="Alice has a parrot.")

conversation.predict(input="Bob has two cats.")

conversation.predict(input="How many pets do Alice and Bob have?")

This will result in the right-hand conversation in the above image. Without the ConversationChain to keep a conversational memory, the conversation will look like the one on the left-hand side in the above image.

Agents: Accessing other tools

Despite being quite powerful, LLMs have some limitations: they lack contextual information (e.g., access to specific knowledge not contained in training data), they can become outdated quickly (e.g., GPT-4 was trained on data before September 2021), and they are bad at math.

Because LLMs can hallucinate on tasks they can’t accomplish on their own, we need to give them access to supplementary tools such as search (e.g., Google Search), calculators (e.g., Python REPL or Wolfram Alpha), and lookups (e.g., Wikipedia).

Additionally, we need agents that make decisions about which tools to use to accomplish a task based on the LLM’s output.


Note that some LLMs such as google/flan-t5-xl won’t be suitable for the following examples as they do not follow the conversational-react-description template. For me, this was the point where I set up a paid account on OpenAI and switched to the OpenAI API.


Below is an example in which the agent first looks up the date of Barack Obama’s birth with Wikipedia and then calculates his age in 2022 with a calculator.

# pip install wikipedia
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType

tools = load_tools(["wikipedia", "llm-math"], llm=llm)
agent = initialize_agent(tools, 
                         llm, 
                         agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, 
                         verbose=True)

agent.run("When was Barack Obama born? How old was he in 2022?")
Output of the LLM agent (Screenshot by the author)
Output of the LLM agent (Screenshot by the author)

Summary

Just a few months ago, we all (or at least most of us) were impressed by ChatGPT’s capabilities. Now, new developer tools like LangChain enable us to build similarly impressive prototypes on our laptops within a few hours – these are some truly exciting times!

LangChain is an open-source Python library that enables anyone who can write code to build LLM-powered applications. The package provides a generic interface to many foundation models, enables prompt management, and acts as a central interface to other components like prompt templates, other LLMs, external data, and other tools via agents – at the time of writing.

The library offers many more features than mentioned in this article. With the current speed of developments, this article could also be potentially outdated in a month.

While working on this article, I noticed that the library and documentation are centered around OpenAI’s API. Although many examples work with the open-source foundation model [google/flan-t5-xl](https://github.com/hwchase17/langchain/issues/1358), I switched to the OpenAI API in between. Despite not being free, experimenting with the OpenAI API in this article cost me only about $1.

Enjoyed This Story?

Subscribe for free to get notified when I publish a new story.

Get an email whenever Leonie Monigatti publishes.

Find me on LinkedIn, Twitter, and Kaggle!

References

[1] Harrison Chase (2023). LangChain documentation (accessed April 23rd, 2023)


Related Articles