
Since the release of ChatGPT, large language models (LLMs) have gained a lot of popularity. Although you probably don’t have enough money and computational resources to train an LLM from scratch in your basement, you can still use pre-trained LLMs to build something cool, such as:
- Personal assistants that can interact with the outside world based on your data
- Chatbots customized for your purpose
- Analysis or summarization of your documents or code
LLMs are changing how we build AI-powered products
With their weird APIs and prompt engineering, LLMs are changing how we build AI-powered products. That’s why new developer tools are emerging everywhere under the term "LLMOps".
One of these new tools is LangChain.
GitHub – hwchase17/langchain: ⚡ Building applications with LLMs through composability ⚡
What is LangChain?
LangChain is a framework built to help you build LLM-powered applications more easily by providing you with the following:
- a generic interface to a variety of different foundation models (see Models),
- a framework to help you manage your prompts (see Prompts), and
- a central interface to long-term memory (see Memory), external data (see Indexes), other LLMs (see Chains), and other agents for tasks an LLM is not able to handle (e.g., calculations or search) (see Agents).
It is an open-source project (GitHub repository) created by Harrison Chase.
Because LangChain has a lot of different functionalities, it may be challenging to understand what it does at first. That’s why we will go over the (currently) six key modules of LangChain in this article to give you a better understanding of its capabilities.
Prerequisites
To follow along in this tutorial, you will need to have the langchain
Python package installed and all relevant API keys ready to use.
Installing LangChain
Before installing the langchain
package, ensure you have a Python version of ≥ 3.8.1 and <4.0.
To install the langchain
Python package, you can pip
install it.
pip install langchain
In this tutorial, we are using version 0.0.147. The GitHub repository is very active; thus, ensure you have a current version.
Once you are all setup, import the langchain
Python package.
import langchain
API keys
Building an application with LLMs requires API keys for some services you want to use, and some APIs have associated costs.
LLM provider (required) – You will first need an API key for the LLM provider you want to use. We are currently experiencing "AI’s Linux moment" where developers must choose between proprietary or open-source foundation models based on a trade-off mainly between performance and cost.

Proprietary models are closed-source foundation models owned by companies with large expert teams and big AI budgets. They usually are larger than open-source models and thus have better performance, but they also have expensive APIs. Examples of proprietary model providers are OpenAI, co:here, AI21 Labs, or Anthropic.
Most available LangChain tutorials use OpenAI but note that the OpenAI API (is not expensive for experimentation but it) is not free. To obtain an OpenAI API Key, you need an OpenAI account and then "Create new secret key" under API keys.
import os
os.environ["OPENAI_API_KEY"] = ... # insert your API_TOKEN here
Open-source models are usually smaller models with lower capabilities than proprietary models, but they are more cost-effective than proprietary ones. Examples of open-source models are:
Many open-source models are organized and hosted on Hugging Face as a community hub. To obtain a Hugging Face API Key, you need a Hugging Face account and create a "New token" under Access Tokens.
import os
os.environ["HUGGINGFACEHUB_API_TOKEN"] = ... # insert your API_TOKEN here
You can use Hugging Face for free for open-source LLMs, but you will be limited to smaller LLMs with less performance.
A personal note – Let’s be honest here for a second: Of course, you can experiment with open-source foundation models here. I tried to make this tutorial only with open-source models hosted on Hugging Face available with a regular account (google/flan-t5-xl and sentence-transformers/all-MiniLM-L6-v2). It works for most examples, but it is also a pain to get some examples to work. Finally, I pulled the trigger and set up a paid account for OpenAI as most examples for LangChain seem to be optimized for OpenAI’s API. Overall running a few experiments for this tutorial cost me about $1.
Vector Database (optional) – If you want to use a specific vector database such as Pinecone, Weaviate, or Milvus, you need to register with them to obtain an API key and check their pricing. In this tutorial, we are using Faiss, which requires no sign-up.
Tools (optional) – Depending on the tools you want the LLM to interact with such as OpenWeatherMap or SerpAPI, you may need to register with them to obtain an API key and check their pricing. In this tutorial, we only use tools requiring no API key.
What can you do with LangChain?
The package provides a generic interface to many foundation models, enables prompt management, and acts as a central interface to other components like prompt templates, other LLMs, external data, and other tools via agents.
At the time of writing, LangChain (version 0.0.147) covers six modules:
- Models: Choosing from different LLMs and embedding models
- Prompts: Managing LLM inputs
- Chains: Combining LLMs with other components
- Indexes: Accessing external data
- Memory: Remembering previous conversations
- Agents: Accessing other tools
The code examples in the following sections are copied and modified from the LangChain documentation.
Models: Choosing from different LLMs and embedding models
Currently, many different LLMs are emerging. LangChain offers integrations to a wide range of models and a streamlined interface to all of them.
LangChain differentiates between three types of models that differ in their inputs and outputs:
- LLMs take a string as an input (prompt) and output a string (completion).
# Proprietary LLM from e.g. OpenAI
# pip install openai
from langchain.llms import OpenAI
llm = OpenAI(model_name="text-davinci-003")
# Alternatively, open-source LLM hosted on Hugging Face
# pip install huggingface_hub
from langchain import HuggingFaceHub
llm = HuggingFaceHub(repo_id = "google/flan-t5-xl")
# The LLM takes a prompt as an input and outputs a completion
prompt = "Alice has a parrot. What animal is Alice's pet?"
completion = llm(prompt)

- Chat models are similar to LLMs. They take a list of chat messages as input and return a chat message.
- Text embedding models take text input and return a list of floats (embeddings), which are the numerical representation of the input text. Embeddings help extract information from a text. This information can then be later used, e.g., for calculating similarities between texts (e.g., movie summaries).
# Proprietary text embedding model from e.g. OpenAI
# pip install tiktoken
from langchain.embeddings import OpenAIEmbeddings
embeddings = OpenAIEmbeddings()
# Alternatively, open-source text embedding model hosted on Hugging Face
# pip install sentence_transformers
from langchain.embeddings import HuggingFaceEmbeddings
embeddings = HuggingFaceEmbeddings(model_name = "sentence-transformers/all-MiniLM-L6-v2")
# The embeddings model takes a text as an input and outputs a list of floats
text = "Alice has a parrot. What animal is Alice's pet?"
text_embedding = embeddings.embed_query(text)

Prompts: Managing LLM inputs
LLMs have weird APIs. Although inputting prompts to LLMs in natural language should feel intuitive, it takes quite a bit of tweaking of the prompt until you get the desired output from an LLM. This process is called prompt engineering.
Once you have a good prompt, you may want to use it as a template for other purposes. Thus, LangChain provides you with so-called PromptTemplates
, which help you construct prompts from multiple components.
from langchain import PromptTemplate
template = "What is a good name for a company that makes {product}?"
prompt = PromptTemplate(
input_variables=["product"],
template=template,
)
prompt.format(product="colorful socks")
The above prompt can be viewed as a zero-shot problem setting, where you hope the LLM was trained on enough relevant data to provide a satisfactory response.
Another trick to improve the LLM’s output is to add a few examples in the prompt and make it a few-shot problem setting.
from langchain import PromptTemplate, FewShotPromptTemplate
examples = [
{"word": "happy", "antonym": "sad"},
{"word": "tall", "antonym": "short"},
]
example_template = """
Word: {word}
Antonym: {antonym}n
"""
example_prompt = PromptTemplate(
input_variables=["word", "antonym"],
template=example_template,
)
few_shot_prompt = FewShotPromptTemplate(
examples=examples,
example_prompt=example_prompt,
prefix="Give the antonym of every input",
suffix="Word: {input}nAntonym:",
input_variables=["input"],
example_separator="n",
)
few_shot_prompt.format(input="big")
The above code will generate a prompt template and compose the following prompt based on the provided examples and input:
Give the antonym of every input
Word: happy
Antonym: sad
Word: tall
Antonym: short
Word: big
Antonym:
Chains: Combining LLMs with other components
Chaining in LangChain simply describes the process of combining LLMs with other components to create an application. Some examples are:
- Combining LLMs with prompt templates (see this section)
- Combining multiple LLMs sequentially by taking the first LLM’s output as the input for the second LLM (see this section)
- Combining LLMs with external data, e.g., for question answering (see Indexes)
- Combining LLMs with long-term memory, e.g., for chat history (see Memory)
In the previous section, we created a prompt template. When we want to use it with our LLM, we can use an LLMChain
as follows:
from langchain.chains import LLMChain
chain = LLMChain(llm = llm,
prompt = prompt)
# Run the chain only specifying the input variable.
chain.run("colorful socks")
If we want to use the output of this first LLM as the input for a second LLM, we can use a SimpleSequentialChain
:
from langchain.chains import LLMChain, SimpleSequentialChain
# Define the first chain as in the previous code example
# ...
# Create a second chain with a prompt template and an LLM
second_prompt = PromptTemplate(
input_variables=["company_name"],
template="Write a catchphrase for the following company: {company_name}",
)
chain_two = LLMChain(llm=llm, prompt=second_prompt)
# Combine the first and the second chain
overall_chain = SimpleSequentialChain(chains=[chain, chain_two], verbose=True)
# Run the chain specifying only the input variable for the first chain.
catchphrase = overall_chain.run("colorful socks")

Indexes: Accessing external data
One limitation of LLMs is their lack of contextual information (e.g., access to some specific documents or emails). You can combat this by giving LLMs access to the specific external data.
For this, you first need to load the external data with a document loader. LangChain provides a variety of loaders for different types of documents ranging from PDFs and emails to websites and YouTube videos.
Let’s load some external data from a YouTube video. You can refer to the official documentation if you want to load a large text document and split it with a Text Splitter.
# pip install youtube-transcript-api
# pip install pytube
from langchain.document_loaders import YoutubeLoader
loader = YoutubeLoader.from_youtube_url("https://www.youtube.com/watch?v=dQw4w9WgXcQ")
documents = loader.load()
Now that you have your external data ready to go as documents
, you can index it with a text embedding model (see Models) in a vector database – a VectorStore. Popular vector databases include Pinecone, Weaviate, and Milvus. In this article, we are using Faiss because it doesn’t require an API key.
# pip install faiss-cpu
from langchain.vectorstores import FAISS
# create the vectorestore to use as the index
db = FAISS.from_documents(documents, embeddings)
Your document (in this case, a video) is now stored as embeddings in a vector store.
Now you can do a variety of things with this external data. Let’s use it for a question-answering task with an information retriever:
from langchain.chains import RetrievalQA
retriever = db.as_retriever()
qa = RetrievalQA.from_chain_type(
llm=llm,
chain_type="stuff",
retriever=retriever,
return_source_documents=True)
query = "What am I never going to do?"
result = qa({"query": query})
print(result['result'])

Wait a minute – Did you just get rickrolled? Yes, you did.
Memory: Remembering previous conversations
For applications like chatbots, it is essential that they can remember previous conversations. But by default, LLMs don’t have any long-term memory unless you input the chat history.

LangChain solves this problem by providing several different options for dealing with chat history :
- keep all conversations,
- keep the latest k conversations,
- summarize the conversation.
In this example, we will use a ConversationChain
to give this application conversational memory.
from langchain import ConversationChain
conversation = ConversationChain(llm=llm, verbose=True)
conversation.predict(input="Alice has a parrot.")
conversation.predict(input="Bob has two cats.")
conversation.predict(input="How many pets do Alice and Bob have?")
This will result in the right-hand conversation in the above image. Without the ConversationChain
to keep a conversational memory, the conversation will look like the one on the left-hand side in the above image.
Agents: Accessing other tools
Despite being quite powerful, LLMs have some limitations: they lack contextual information (e.g., access to specific knowledge not contained in training data), they can become outdated quickly (e.g., GPT-4 was trained on data before September 2021), and they are bad at math.
Because LLMs can hallucinate on tasks they can’t accomplish on their own, we need to give them access to supplementary tools such as search (e.g., Google Search), calculators (e.g., Python REPL or Wolfram Alpha), and lookups (e.g., Wikipedia).
Additionally, we need agents that make decisions about which tools to use to accomplish a task based on the LLM’s output.
Note that some LLMs such as google/flan-t5-xl
won’t be suitable for the following examples as they do not follow the conversational-react-description
template. For me, this was the point where I set up a paid account on OpenAI and switched to the OpenAI API.
Below is an example in which the agent first looks up the date of Barack Obama’s birth with Wikipedia and then calculates his age in 2022 with a calculator.
# pip install wikipedia
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
tools = load_tools(["wikipedia", "llm-math"], llm=llm)
agent = initialize_agent(tools,
llm,
agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
verbose=True)
agent.run("When was Barack Obama born? How old was he in 2022?")

Summary
Just a few months ago, we all (or at least most of us) were impressed by ChatGPT’s capabilities. Now, new developer tools like LangChain enable us to build similarly impressive prototypes on our laptops within a few hours – these are some truly exciting times!
LangChain is an open-source Python library that enables anyone who can write code to build LLM-powered applications. The package provides a generic interface to many foundation models, enables prompt management, and acts as a central interface to other components like prompt templates, other LLMs, external data, and other tools via agents – at the time of writing.
The library offers many more features than mentioned in this article. With the current speed of developments, this article could also be potentially outdated in a month.
While working on this article, I noticed that the library and documentation are centered around OpenAI’s API. Although many examples work with the open-source foundation model [google/flan-t5-xl](https://github.com/hwchase17/langchain/issues/1358)
, I switched to the OpenAI API in between. Despite not being free, experimenting with the OpenAI API in this article cost me only about $1.
Enjoyed This Story?
Subscribe for free to get notified when I publish a new story.
Find me on LinkedIn, Twitter, and Kaggle!
References
[1] Harrison Chase (2023). LangChain documentation (accessed April 23rd, 2023)