#LLM FOR BEGINNERS
Using LangChain ReAct Agents for Answering Multi-hop Questions in RAG Systems
Useful when answering complex queries on internal documents in a step-by-step manner with ReAct and Open AI Tools agents.
Published in
42 min readFeb 15, 2024
Motivation
The basic RAG chatbots I have built in the past using standard LangChain components such as vectorstore, retrievers, etc have worked out well for me. Depending on the internal dataset I feed in, they are capable of handling humble questions such as “What is the parental leave policy in India” (source dataset: HR policy documents), “What are the main concerns regarding the flavor of our product” (source dataset: social media/Tweets), “What are the themes in Monet paintings” (source dataset: Art Journals), etc. More recently, the complexity of queries being fed to it has increased, for instance, “Has there been an increase in the concerns regarding flavor in the past 1 month”. Unless there is a specific section in the internal documents that specifically talks about the comparison, it is highly unlikely the chatbot would display the correct answer. The reason is — the correct answer requires the following steps to be planned/executed systematically:
- STEP 1: calculate the start date and end date based on “past 1 month” and today’s date
- STEP 2: fetch the queries mentioning flavor issues for the start date
- STEP 3: count the queries from Step 2
- STEP 4: fetch the queries mentioning flavor issues for the end date
- STEP 5: count the queries from Step 4
- STEP 6: calculate the percentage increase/decrease using counts from Step 3 and Step 5.
Luckily for us, LLMs are very good at such planning! And Langchain agents are the ones orchestrating this planning for us.
The core idea of agents is to use a language model to choose a sequence of actions to take. In agents, a language model is used as a reasoning engine to determine which actions to take and in which order. [Source]
Agents typically require a set of tools to be specified at the time of their instantiation. For instance, to solve the aforementioned multi-hop question, we should define four tools:
Tool_Date: A Python function that takes as input a relative time frame (such as the past 6 months) and calculates the start date by subtracting the time frame from today’s date (for Step#1)Tool_Search: A search engine that can take as input a search term and return the list of relevant documents (for Step#2 and Step#4)Tool_Length: A Python function that takes as input a list and returns the length of that list (for Step#3 and Step#5)Tool_PercCalculator: A Python function that takes as input two numbers and returns the percent change calculation (for Step#6)
Generally speaking — be mindful of the choice of tools you provide an agent, as these are the only tools that the agent will use to answer each of the intermediate steps. If it finds a relevant tool — great, it will use it to get the answer. If it doesn’t, it will usually iterate a few times (i.e. trying one of the other available tools or its own logical reasoning) and finally return a sub-optimal answer.
Let’s begin by jumping straight into code, if you’d like to follow along, here’s the GitHub repo.
Dataset
While I was tempted to use the ever-popular state_of_the_union.txt for this demo, I couldn’t come up with complex questions to ask that document. Hence, I have created a dummy HR document (using ChatGPT) for a fictitious company called GlobalCorp. You can view globalcorp_hr_policy.txt here. The main highlights of the file include (a) country-specific annual budgets (b) in different currencies and (c) country-specific leave policies.
LLM and Embedding models
We will be using Azure Open AI models (gpt3.5 turbo , gpt-4-turbo and ada-embeddings) for this tutorial.
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
OPENAI_DEPLOYMENT_NAME = "gpt-35-turbo-16k"
OPENAI_DEPLOYMENT_ENDPOINT = "https://<???>.openai.azure.com/"
OPENAI_DEPLOYMENT_VERSION = "2023-12-01-preview"
OPENAI_MODEL_NAME = "gpt-35-turbo-16k"
OPENAI_ADA_EMBEDDING_DEPLOYMENT_NAME = "text-embedding-ada"
OPENAI_ADA_EMBEDDING_MODEL_NAME = "text-embedding-ada-002"
llm = AzureChatOpenAI(
deployment_name=OPENAI_DEPLOYMENT_NAME,
model_name=OPENAI_MODEL_NAME,
openai_api_base=OPENAI_DEPLOYMENT_ENDPOINT,
openai_api_version=OPENAI_DEPLOYMENT_VERSION,
openai_api_key=OPENAI_API_KEY,
openai_api_type="azure",
temperature=0.1,
)
embeddings = OpenAIEmbeddings(
deployment=OPENAI_ADA_EMBEDDING_DEPLOYMENT_NAME,
model=OPENAI_ADA_EMBEDDING_MODEL_NAME,
openai_api_base=OPENAI_DEPLOYMENT_ENDPOINT,
openai_api_type="azure",
chunk_size=1,
openai_api_key=OPENAI_API_KEY,
openai_api_version=OPENAI_DEPLOYMENT_VERSION,
)Disclaimer — I will be using the terms “RAG tool”, “Q&A system”, and “QnA tool” interchangeably. For this tutorial, all refer to a tool that is capable of looking up a bunch of documents to answer a specific user query but does not have any conversational memory i.e. you won’t be able to ask follow-up questions in a chat-like manner. However, that can be easily implemented in LangChain and will likely be covered in some future article. The focus here is just to get the multi-hop questions working.
RAG-based QnA
Let’s go ahead and build a standard Q&A system using this data.
We will be using TextLoader for loading the dummy HR document.
# Loading the document
from langchain.document_loaders import TextLoader
loader = TextLoader("../data/globalcorp_hr_policy.txt")
documents = loader.load()Chroma as the vectorstore (for storing the document embeddings),
persist_directory = "local_vectorstore"
collection_name = "hrpolicy"
PROJECT_ROOT = "...." #insert your project root directory name here
vectorstore = Chroma(
persist_directory=os.path.join(PROJECT_ROOT, "data", persist_directory),
collection_name=collection_name,
embedding_function=embeddings,
)LocalFileStore as the docstore (for storing the parent documents),
from langchain.storage._lc_store import create_kv_docstore
# The storage layer for the parent documents
local_store = "local_docstore"
local_store = LocalFileStore(os.path.join(PROJECT_ROOT, "data", local_store))
docstore = create_kv_docstore(local_store)PDR (parentdocumentretriever) as the retriever (for retrieving relevant data from the index).
# This text splitter is used to create the child documents
parent_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=20)
child_splitter = RecursiveCharacterTextSplitter(chunk_size=200, chunk_overlap=20)
retriever = ParentDocumentRetriever(
vectorstore=vectorstore,
docstore=docstore,
child_splitter=child_splitter,
parent_splitter=parent_splitter,
)With all the setup done, we are ready to add our document to the retriever using add_documents() command. Additionally, I also recommend persisting the vectorstore using .persist() command (i.e. storing the contents of the vectorstore to your disk so you don’t have to recompute the embeddings again once you terminate your current session)
# run only once
# vectorstore.persist()
# retriever.add_documents(documents, ids=None)Once you run these commands, you should see two folders — local_docstore and local_vectorstore — created in your working session. Feel free to check the contents for each.
Quick sanity check to see if the retriever is set correctly:
retriever.get_relevant_documents("communication initiatives?")
## OUTPUT ##
[Document(page_content='**Health and Safety:**\nWorkplace safety is a shared responsibility. Emergency procedures are clearly posted throughout our office buildings, and an annual budget of $10,000, €5,000, and ¥1 million is allocated for safety drills and equipment maintenance in the U.S., Germany, and Japan, respectively.\n\n**Communication:**\nImportant updates are conveyed through company-wide emails and team meetings. An annual budget of $500,000, €250,000, and ¥25 million is allocated for communication initiatives, including employee engagement events in the U.S., Germany, and Japan, respectively.\n\nThis policy undergoes an annual review to ensure relevance and compliance. Welcome to GlobalCorp, where our commitment to a diverse, inclusive, and respectful workplace is the foundation of our success.\n\n1.\tRecruitment and Selection\n1.\tIntroduction', metadata={'source': '../data/globalcorp_hr_policy.txt'}),
Document(page_content='•\ttwo satisfactory references;\n•\tconfirmation of the right to work in this country (if appropriate)\n•\tCriminal Records Disclosure (if appropriate).\n6.\tProbationary Period\n\t\n6.1\tAll appointments into the Company will be made subject to a probationary period of six calendar months. After three months a review meeting will take place between the post holder and their line manager to discuss progress. At the end of the probationary period, and subject to a satisfactory report by the appropriate head of section or line manager, employees will be notified in writing that they have successfully completed their probationary period. The probationary period can be extended by a further 3 months should the individuals line manager consider this appropriate.\n7.\tRecruitment Monitoring', metadata={'source': '../data/globalcorp_hr_policy.txt'})]Finally, we’ll build the RetrievalQA chain to do question-answering using all the aforementioned components.
from langchain.chains import RetrievalQA
qa = RetrievalQA.from_chain_type(
llm=llm,
chain_type="stuff",
retriever=retriever,
return_source_documents=True,
)Asking standard questions
qa({"query": "What is the allocated budget for communication initiatives?"})
## OUTPUT ##
{'query': 'What is the allocated budget for communication initiatives?',
'result': 'The allocated budget for communication initiatives is $500,000 in the United States, €250,000 in Germany, and ¥25 million in Japan.',
'source_documents': [Document(page_content='**Health and Safety:**\nWorkplace safety is a shared responsibility. Emergency procedures are clearly posted throughout our office buildings, and an annual budget of $10,000, €5,000, and ¥1 million is allocated for safety drills and equipment maintenance in the U.S., Germany, and Japan, respectively.\n\n**Communication:**\nImportant updates are conveyed through company-wide emails and team meetings. An annual budget of $500,000, €250,000, and ¥25 million is allocated for communication initiatives, including employee engagement events in the U.S., Germany, and Japan, respectively.\n\nThis policy undergoes an annual review to ensure relevance and compliance. Welcome to GlobalCorp, where our commitment to a diverse, inclusive, and respectful workplace is the foundation of our success.\n\n1.\tRecruitment and Selection\n1.\tIntroduction', metadata={'source': '../data/globalcorp_hr_policy.txt'}),
Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}),
Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]}qa({"query": "How many maternity leaves are offered?"})
## OUTPUT ##
{'query': 'What is the probationary period?',
'result': "The probationary period is a period of six calendar months during which an employee's performance and suitability for the job are assessed. At the end of the probationary period, if the employee's performance is satisfactory, they will be notified in writing that they have successfully completed their probationary period. The probationary period can be extended by a further 3 months if the employee's line manager deems it necessary.",
'source_documents': [Document(page_content='•\ttwo satisfactory references;\n•\tconfirmation of the right to work in this country (if appropriate)\n•\tCriminal Records Disclosure (if appropriate).\n6.\tProbationary Period\n\t\n6.1\tAll appointments into the Company will be made subject to a probationary period of six calendar months. After three months a review meeting will take place between the post holder and their line manager to discuss progress. At the end of the probationary period, and subject to a satisfactory report by the appropriate head of section or line manager, employees will be notified in writing that they have successfully completed their probationary period. The probationary period can be extended by a further 3 months should the individuals line manager consider this appropriate.\n7.\tRecruitment Monitoring', metadata={'source': '../data/globalcorp_hr_policy.txt'})]}qa({"query": "What is the difference in the number of work hours in Germany vs. United States?"})
## OUTPUT ##
{'query': 'What is the difference in the number of work hours in Germany vs. United States?',
'result': 'In Germany, the standard workweek is 38 hours (Monday to Friday, 8 AM to 5 PM), while in the United States, employees adhere to a standard 40-hour workweek (Monday to Friday, 9 AM to 5 PM). So, the difference in the number of work hours between Germany and the United States is 2 hours per week.',
'source_documents': [Document(page_content='**GlobalCorp Human Resources Policy**\n\nWelcome to GlobalCorp, where our Human Resources Policy is designed to provide a comprehensive framework for employees across our offices in the United States, Germany, and Japan. We operate under an at-will employment relationship, and any contractual agreements should be documented in writing.\n\nAt the core of our culture is a commitment to professionalism and ethical conduct. Clear and respectful communication is highly valued, and a business casual dress code is encouraged.\n\n**Work Hours:**\nEmployees in the United States adhere to a standard 40-hour workweek (Monday to Friday, 9 AM to 5 PM). In Germany, the standard workweek is 38 hours (Monday to Friday, 8 AM to 5 PM), and in Japan, employees work 40 hours per week (Monday to Friday, 9 AM to 6 PM). Punctuality is paramount, and employees are expected to arrive on time. Time-off requests follow country-specific guidelines.', metadata={'source': '../data/globalcorp_hr_policy.txt'}),
Document(page_content='**Leave Policies - Germany:**\nIn Germany, generous leave policies offer 30 days of paid vacation and 20 days of paid sick leave annually. An annual budget of €1 million is allocated for leave-related expenses.\n\n**Leave Policies - Japan:**\nIn Japan, employees enjoy 20 days of paid vacation and 15 days of paid sick leave per year. An annual budget of ¥100 million is allocated for leave-related expenses.\n\n**Performance Management:**\nPerformance reviews are conducted annually, with regular feedback provided to support professional development. GlobalCorp encourages continuous learning and allocates an annual budget of $5,000 per employee for training and development opportunities.', metadata={'source': '../data/globalcorp_hr_policy.txt'}),
Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]}Having manually reviewed the policy document, it is safe to say the answers make sense.
Asking complex/multi-hop questions
# incorrect as the currency conversion used is wrong. We need to fix this!
qa({"query": "What is the percentage difference in the annual budget for Japan and US?"})
## OUTPUT ##
{'query': 'What is the percentage difference in the annual budget for Japan and US?',
'result': 'The annual budget for Japan is ¥50 million, and the annual budget for the United States is $500,000. To calculate the percentage difference, we need to convert the budgets to the same currency. Assuming an exchange rate of 1 USD = 100 JPY, the converted budget for Japan is ¥50 million = $500,000. \n\nThe percentage difference can be calculated as follows:\n\nPercentage Difference = ((Budget for Japan - Budget for US) / Budget for US) * 100\n\n= (($500,000 - $500,000) / $500,000) * 100\n\n= (0 / $500,000) * 100\n\n= 0%\n\nTherefore, the percentage difference in the annual budget for Japan and the United States is 0%.',
'source_documents': [Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}),
Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'}),
Document(page_content='**Leave Policies - Germany:**\nIn Germany, generous leave policies offer 30 days of paid vacation and 20 days of paid sick leave annually. An annual budget of €1 million is allocated for leave-related expenses.\n\n**Leave Policies - Japan:**\nIn Japan, employees enjoy 20 days of paid vacation and 15 days of paid sick leave per year. An annual budget of ¥100 million is allocated for leave-related expenses.\n\n**Performance Management:**\nPerformance reviews are conducted annually, with regular feedback provided to support professional development. GlobalCorp encourages continuous learning and allocates an annual budget of $5,000 per employee for training and development opportunities.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]}Mathematically speaking, the response is not 100% correct. Even though the logic it used is correct (i.e. converting the amount from ¥ to $ ), the exchange rate used is out of date.
Let’s try helping it by providing the exchange rate information (1 USD = 147.72 JPY) as part of the query itself.
# Results are still slightly off!
qa({"query": "What is the percentage difference in the annual budget for Japan and US if 1 USD = 147.72 JPY?"})
## OUTPUT ##
{'query': 'What is the percentage difference in the annual budget for Japan and US if 1 USD = 147.72 JPY?',
'result': 'To calculate the percentage difference in the annual budget for Japan and the United States, we need to convert the budgets from their respective currencies to a common currency, such as USD. \n\nGiven that 1 USD = 147.72 JPY, we can convert the annual budget for Japan from JPY to USD by dividing it by the exchange rate:\n\nAnnual budget for Japan in USD = ¥50,000,000 / 147.72 = $338,164.25\n\nNow we can calculate the percentage difference between the annual budgets for Japan and the United States:\n\nPercentage difference = ((Annual budget for Japan - Annual budget for the United States) / Annual budget for the United States) * 100\n\nPercentage difference = (($338,164.25 - $1,000,000) / $1,000,000) * 100\n\nPercentage difference = (-$661,835.75 / $1,000,000) * 100\n\nPercentage difference = -66.18%\n\nTherefore, the percentage difference in the annual budget for Japan and the United States is approximately -66.18%.',
'source_documents': [Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}),
Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'}),
Document(page_content='**Leave Policies - Germany:**\nIn Germany, generous leave policies offer 30 days of paid vacation and 20 days of paid sick leave annually. An annual budget of €1 million is allocated for leave-related expenses.\n\n**Leave Policies - Japan:**\nIn Japan, employees enjoy 20 days of paid vacation and 15 days of paid sick leave per year. An annual budget of ¥100 million is allocated for leave-related expenses.\n\n**Performance Management:**\nPerformance reviews are conducted annually, with regular feedback provided to support professional development. GlobalCorp encourages continuous learning and allocates an annual budget of $5,000 per employee for training and development opportunities.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]}Interestingly enough, LLM was able to use the exchange rate as part of the calculations and the answer it gave (i.e. $338,164.25) was very close to the actual answer (i.e. 338,478.20). Having said that, there’s still room for improvement.
Let’s try another question, this time a comparison question:
# incorrect as technically US has higher budget after conversion
qa({"query": "Which country has the highest budget?"})
## OUTPUT ##
{'query': 'Which country has the highest budget?',
'result': 'Japan has the highest budget for employee benefits, with an annual allocation of ¥50 million.',
'source_documents': [Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}),
Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]}Again, the response is not correct since it didn’t do the currency conversion before comparing the budgets with other countries like the US, Germany, etc.
Observation: All the aforementioned questions could be reliably answered if instead of jumping on a final answer, we had a systematic way of planning the intermediate steps. To do so, let’s introduce agents.
ReAct Agent
In this tutorial, we will be using LangChain’s implementation of the ReAct (Reason + Act) agent, first introduced in this paper. The key takeaway from the paper is that if we prompt the LLM to generate both reasoning traces and task-specific actions in a step-by-step manner, its performance on the task improves. In other words, we are explicitly asking it to have multiple thought-action-observation steps to solve a task instance instead of coming to the final answer in one single jump (which ultimately leads to reduced hallucination).
Apart from ReAct, LangChain supports other agents such as Open AI tools, XML, StructuredChat, Self Ask with Search, etc that I strongly encourage you to read about here. One key thing to note here is that ReAct agents can only support tools that can take only 1 input parameter (for instance, from the tools described above, it can support Tool_Length, Tool_Date, and Tool_Search ). If you want to use tools that take more than 1 input (for instance Tool_PercCalculator), you will be better off using Open AI Tools agent or Open AI Functions agent.
Note: “OpenAI termed the capability to invoke a single function as
functions, and the capability to invoke one or more functions astools. As per the official website,functionsare now considered a legacy option that is deprecated in favor oftools. So if you’re creating agents using OpenAI models, you should be using thisOpenAI Toolsagent rather than theOpenAI functionsagent.” [Source]
Defining the tools for the agent
As I mentioned above, we first need to define the tools that this agent will have access to. For starters, we will only define one tool: tool_search.
tool_search: Given a search query, we need a tool that will return the relevant chunks of the HR document. But wait, isn’t that what our PDR retriever does anyway? In fact, we can easily convert our retriever into a tool using the create_retriever_tool().
from langchain.tools.retriever import create_retriever_tool
tool_search = create_retriever_tool(
retriever=retriever,
name="search_hr_policy",
description="Searches and returns excerpts from the HR policy.",
)A couple of pointers:
- The
nameanddescriptionof the tool will be passed in the API call to the LLM, so make sure it is as unambiguous as possible for the agent to understand. In our case, we have clearly defined that this tool returns excerpts (i.e. chunks) from the HR policy. - Under the hood, this tool uses the
get_relevant_documents()function of theretriever. You can check it using.func:
tool_search.func
## Output ##
<bound method BaseRetriever.get_relevant_documents of ParentDocumentRetriever(vectorstore=<langchain_community.vectorstores.chroma.Chroma object at 0x11f0c5f10>, docstore=<langchain.storage.encoder_backed.EncoderBackedStore object at 0x11f0c5f40>, child_splitter=<langchain.text_splitter.RecursiveCharacterTextSplitter object at 0x11f0c5430>)>- You can even check the schema of this tool using
.schema(). It is useful for verifying therequiredparameters necessary for calling the tool.
tool.args_schema.schema()
## Output ##
{'title': 'RetrieverInput',
'description': 'Input to the retriever.',
'type': 'object',
'properties': {'query': {'title': 'Query',
'description': 'query to look up in retriever',
'type': 'string'}},
'required': ['query']}Pro-tip: You can use tool.invoke({"inp_param_name": inp_param_value}) to quickly test if a custom tool has been set up properly. For example: tool_search.invoke({“query”: “enter query here”})
Finally, let’s set up the ReAct agent using a prompt that emphasizes multiple thought-action-observation steps. Luckily for us, this is already available on the LangChain hub (you can also override this by defining your own). The prompt template requires three input_variables i.e. tools, input and agent_scratchpad.
from langchain import hub
prompt = hub.pull("hwchase17/react")
print(prompt.template)
## OUTPUT ##
Answer the following questions as best you can. You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin!
Question: {input}
Thought:{agent_scratchpad}Note 1: While this prompt works fine 8 out of 10 times, I highly recommend modifying it to suit your use case — especially if the agent is getting confused about the sequence of actions or thoughts. For instance, in one of the projects where the questions were comparison-based (such as ‘Compare the increase in sales between China and the US in the last 1 year’), this is how I updated the react prompt (and introduced a new input variable {today_date}:
Use the context given to you to answer the question.
When you come across time based questions like Has there been increase in sales from April last year and August last year,
use today's date as {today_date} for fetching exact start and end dates. Then your first action should be to fetch the sales data with metadata time between April 1st to April 30th.
That will be your answer 1. Your next action will be to fetch the sales data with metadata time between August 1st to
August 31st, this will be your answer 2. Now analyse and understand each of answer 1 and answer 2, calculate the
percentange change, compare them and answer the question.
You have access to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought: you should always think about what to do
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought: I now know the final answer
Final Answer: the final answer to the original input question
Begin!
Question: {input}
Thought:{agent_scratchpad}Note 2: Doing the above (i.e. a more detailed/lengthy prompt) can exponentially increase the token count for your use case. My recommendation is to switch to the chaining of LLMs if (and this is a big if) you know beforehand the order in which the action/thoughts need to be executed. For example: if you know that the only type of questions your system needs to answer is comparison questions like the one I mentioned above, it makes more sense to create a sequential chain of calls to the LLM where output from one is fed as input to the other. I have touched upon the technical implementation of this in LangChain in my previous article.
Creating a ReAct agent
# only creates the logical steps for us
react_agent = create_react_agent(llm, [tool_search], prompt)We also need to instantiate AgentExecutor that will execute the logical steps that react_agent will generate.
# executes the logical steps we created
agent_executor = AgentExecutor(
agent=react_agent,
tools=[tool],
verbose=True,
handle_parsing_errors=True,
max_iterations = 5 # useful when agent is stuck in a loop
)Testing the ReAct agent
Finally, it is time to test it on the same sample queries as before.
Note: While I was tempted to cherry-pick examples for this article, it is important to show that these agents can be unreliable at times and a lot of testing needs to be done to establish their capabilities and limitations.
query = "Which country has the highest budget?"
agent_executor.invoke({"input": query})
# if you are using the updated prompt with today's date passed to it
# agent_executor.invoke({"input": query, "today_date": date.today()})
## OUTPUT ##
> Entering new AgentExecutor chain...
I don't have access to information about country budgets. I should try searching for this information.
Action: search_hr_policy
Action Input: "highest country budget"[Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Health and Safety:**\nWorkplace safety is a shared responsibility. Emergency procedures are clearly posted throughout our office buildings, and an annual budget of $10,000, €5,000, and ¥1 million is allocated for safety drills and equipment maintenance in the U.S., Germany, and Japan, respectively.\n\n**Communication:**\nImportant updates are conveyed through company-wide emails and team meetings. An annual budget of $500,000, €250,000, and ¥25 million is allocated for communication initiatives, including employee engagement events in the U.S., Germany, and Japan, respectively.\n\nThis policy undergoes an annual review to ensure relevance and compliance. Welcome to GlobalCorp, where our commitment to a diverse, inclusive, and respectful workplace is the foundation of our success.\n\n1.\tRecruitment and Selection\n1.\tIntroduction', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]I found information about budgets in the HR policy, but it seems to be related to cybersecurity training, safety drills, and communication initiatives. It doesn't provide information about country budgets. I need to search for information about country budgets elsewhere.
Action: search_hr_policy
Action Input: "country budgets"[Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**GlobalCorp Human Resources Policy**\n\nWelcome to GlobalCorp, where our Human Resources Policy is designed to provide a comprehensive framework for employees across our offices in the United States, Germany, and Japan. We operate under an at-will employment relationship, and any contractual agreements should be documented in writing.\n\nAt the core of our culture is a commitment to professionalism and ethical conduct. Clear and respectful communication is highly valued, and a business casual dress code is encouraged.\n\n**Work Hours:**\nEmployees in the United States adhere to a standard 40-hour workweek (Monday to Friday, 9 AM to 5 PM). In Germany, the standard workweek is 38 hours (Monday to Friday, 8 AM to 5 PM), and in Japan, employees work 40 hours per week (Monday to Friday, 9 AM to 6 PM). Punctuality is paramount, and employees are expected to arrive on time. Time-off requests follow country-specific guidelines.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]I still couldn't find information about country budgets in the HR policy. I should try searching for this information using a different tool or source.
Action: search_external_source
Action Input: "highest country budget"search_external_source is not a valid tool, try one of [search_hr_policy].I couldn't find information about country budgets in the HR policy or using the available tools. I need to consult a different source or resource to find the answer to this question.
Final Answer: I don't have access to information about which country has the highest budget.A few points for consideration:
- Notice the
Action Input(i.e. “highest country budget”) for the firstAction. This is the search query that will be passed to theget_relevant_function()of the PDRretriever(rather than the actual input query i.e.Which country has the highest budget?). That means, if there was a section in the underlying documents that talked about the highest country budget we would have been sorted! Sadly, that’s not the case here. - The
Observation(i.e. the results from running theactioni.e. tool with theaction inputs) is printed right afterAction Input. In our case, it is the retrieved document ([Document(page_content=’**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are …) and it contains the information necessary for answering the query. Even so, the final response is still incorrect. [P.S. Based on my testing, this happens mostly with gpt3.5.] - (To make it work with gpt-3.5) I tried updating the search query to
"Which of the two countries has the highest budget — Japan or Unites States?", hoping the agent would pick up on the country names and make 2 consecutive calls to the retriever to fetch the relevant info. Unfortunately, the final answer was the same as above. - Finally, after slight rewording, we have a query that works (with gpt3.5).
Main takeaway: (1) Even for similar-looking prompts, the responses can vary drastically. (2) GPT4 is better suited than GPT3.5 for implementing ReAct agents.
query = "Is the budget for Japan different than United States?"
agent_executor.invoke({"input": query})
## OUTPUT ##
> Entering new AgentExecutor chain...
I should check the HR policy to see if there is any information about budget differences between Japan and the United States.
Action: search_hr_policy
Action Input: "budget differences between Japan and United States"[Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Leave Policies - Germany:**\nIn Germany, generous leave policies offer 30 days of paid vacation and 20 days of paid sick leave annually. An annual budget of €1 million is allocated for leave-related expenses.\n\n**Leave Policies - Japan:**\nIn Japan, employees enjoy 20 days of paid vacation and 15 days of paid sick leave per year. An annual budget of ¥100 million is allocated for leave-related expenses.\n\n**Performance Management:**\nPerformance reviews are conducted annually, with regular feedback provided to support professional development. GlobalCorp encourages continuous learning and allocates an annual budget of $5,000 per employee for training and development opportunities.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]I have found information about budget differences between Japan and the United States in the HR policy.
Action: search_hr_policy
Action Input: "budget differences between Japan and United States"[Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Leave Policies - Germany:**\nIn Germany, generous leave policies offer 30 days of paid vacation and 20 days of paid sick leave annually. An annual budget of €1 million is allocated for leave-related expenses.\n\n**Leave Policies - Japan:**\nIn Japan, employees enjoy 20 days of paid vacation and 15 days of paid sick leave per year. An annual budget of ¥100 million is allocated for leave-related expenses.\n\n**Performance Management:**\nPerformance reviews are conducted annually, with regular feedback provided to support professional development. GlobalCorp encourages continuous learning and allocates an annual budget of $5,000 per employee for training and development opportunities.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]I have found information about budget differences between Japan and the United States in the HR policy.
Action: search_hr_policy
Action Input: "budget differences between Japan and United States"[Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Leave Policies - Germany:**\nIn Germany, generous leave policies offer 30 days of paid vacation and 20 days of paid sick leave annually. An annual budget of €1 million is allocated for leave-related expenses.\n\n**Leave Policies - Japan:**\nIn Japan, employees enjoy 20 days of paid vacation and 15 days of paid sick leave per year. An annual budget of ¥100 million is allocated for leave-related expenses.\n\n**Performance Management:**\nPerformance reviews are conducted annually, with regular feedback provided to support professional development. GlobalCorp encourages continuous learning and allocates an annual budget of $5,000 per employee for training and development opportunities.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]I have found information about budget differences between Japan and the United States in the HR policy.
Action: search_hr_policy
Action Input: "budget differences between Japan and United States"[Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Leave Policies - Germany:**\nIn Germany, generous leave policies offer 30 days of paid vacation and 20 days of paid sick leave annually. An annual budget of €1 million is allocated for leave-related expenses.\n\n**Leave Policies - Japan:**\nIn Japan, employees enjoy 20 days of paid vacation and 15 days of paid sick leave per year. An annual budget of ¥100 million is allocated for leave-related expenses.\n\n**Performance Management:**\nPerformance reviews are conducted annually, with regular feedback provided to support professional development. GlobalCorp encourages continuous learning and allocates an annual budget of $5,000 per employee for training and development opportunities.', metadata={'source': '../data/globalcorp_hr_policy.txt'})]I have found information about budget differences between Japan and the United States in the HR policy.
Final Answer: According to the HR policy, the annual budget for employee benefits in the United States is $1 million, while the budget for Japan is ¥50 million.
> Finished chain.P.S. I was not too pleased with the unnecessary Action-Action Input loops even though the relevant answer was revealed in the first iteration itself. Again, something I am currently debugging.
Understanding the implementation of the agent
LangChain library can be a bit daunting at first and if you would like to debug how things are working under the hood w.r.t. react agents, here are some useful breakpoints to set in your debugger.
I. setup of the ReAct agent: here you will find the 4 main steps (chained together through the | symbol) that the agent will take at each iteration. (I have also included snippets to show you inputs/outputs for each of the four steps in isolation).
P.S. If it is your first time seeing this pipe symbol | in LangChain, I recommend going through this and this first. In simple terms, the |takes passes the output from the first step and passes it as input to the next step in the chain.
(a) Runnable.assign(): updates agent_scratchpad with observations i.e. all prior thought-action-observations and creates a dictionary that can be passed as input to the next step i.e. PromptTemplate.
While I have used dummy data in the snippet below, a typical agent_scratchpad would look like this:`I need to check if there is any information in the HR policy regarding budget allocation for different countries.\nAction: search_hr_policy\nAction Input: “budget allocation for different countries”\nObservation: [Document(page_content=\’**Griev....]metadata={\’source\’: \’../data/globalcorp_hr_policy.txt\’})]\nThought:
# Testing Runnable in isolation
from langchain_core.agents import AgentAction
from langchain_core.runnables import Runnable, RunnablePassthrough
agent_1 = RunnablePassthrough.assign(
agent_scratchpad=lambda x: format_log_to_str(x["intermediate_steps"])
)
input = {
"input": "I love programming",
"intermediate_steps": [
(AgentAction(tool="DummyTool", tool_input="Foo", log="Some log here"), "Result of the tool")
],
}
output_1 = agent_1.invoke(input)
## OUTPUT ##
{'input': 'I love programming',
'intermediate_steps': [(AgentAction(tool='DummyTool', tool_input='Foo', log='Some log here'),
'Result of the tool')],
'agent_scratchpad': 'Some log here\nObservation: Result of the tool\nThought: '}(b) PromptTemplate: frames the final react prompt for the LLM call based on the updated agent_scratchpad and creates a StringPromptValue
(Note: As per the react prompt template, we also need another input_variables called tools which is already appended using prompt.partial here).
# Testing PromptTemplate in isolation
agent_2 = prompt
output_2 = agent_2.invoke(output_1)
## OUTPUT ##
StringPromptValue(text='Answer the following questions as best you can. You have access to the following tools:\n\nsearch_hr_policy: Searches and returns excerpts from the HR policy.\n\nUse the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [search_hr_policy]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin!\n\nQuestion: I love programming\nThought:Some log here\nObservation: Result of the tool\nThought: ')(c ) AzureChatOpenAI: passes the prompt to the llm for the generation step and fetches the response i.e. AIMessage
# Testing llm in isolation
from langchain_core.prompt_values import StringPromptValue
inp = StringPromptValue(text='Repeat everything I say - "Parrot is a bird')
llm_with_stop = llm.bind(stop=["bird"])
agent_3 = llm_with_stop
agent_3.invoke(inp)
## OUTPUT ##
AIMessage(content='Parrot is a ')(d) ReActSingleInputOutputParser(): parses the output (i.e. AIMessage) returned by the llm.
from langchain_core.messages.ai import AIMessage
content = AIMessage(content="I found information about the budget differences between US and Japan in the HR policy. I need to use the currency conversion tool to compare the budgets in USD.\nAction: Currency conversion\nAction Input: ¥50 million")
agent = ReActSingleInputOutputParser()
agent.invoke(content)
## OUTPUT ##
AgentAction(tool='Currency conversion', tool_input='¥50 million', log='I found information about the budget differences between US and Japan in the HR policy. I need to use the currency conversion tool to compare the budgets in USD.\nAction: Currency conversion\nAction Input: ¥50 million')II. agent at work: this is where you can see the for loop for iterating over all four aforementioned steps. Feel free to set watch variables and investigate the intermediate results. After going through all four steps, the final response is either of type AgentAction (whether to call another tool) or AgentFinish(finish the loop). Here is a quick snapshot of my debugger at all the four steps:
III. Deep dive into the parse() of ReactSingleInputOutputParser: if you want to know how it’s decided whether AIMessage should result in AgentAction or AgentFinish.
IV. custom tool being used: this is where you can see your custom tool being used by the agent (if AgentAction was returned).
V. the while loop that keeps the agent looping (unless AgentFinish is encountered or a time-out occurs) after updating the intermediate steps with the observations from the previous iteration.
Note: Intermediate steps are a collection of observations and observations will often be the output of the tool. So in the case of the retriever tool, it will be a list of Documents, in the case of currency conversion, it will be a number, and so on.
Final thoughts: If the questions anticipated for a QnA system are fundamentally basic, meaning they can be adequately handled by a standard retrieval-based QA mechanism without the need for multi-hop reasoning, it’s best to steer clear of agents. This is especially true if the only tool required is a ‘retriever-turned-tool’. Introducing agents in such scenarios can needlessly complicate matters. Moreover, if you are using a retriever as a tool, the input to its get_relevant_function() gets modified by the agent (as you noticed above) as it sees fit and you no longer have control over it. This may be an issue in some cases (although an easy fix for that is to update the description of the tool to tool_search.description = “Searches and returns excerpts from the HR policy. Input should be a fully formed question”)
The true potential of agents is unlocked when we give it complex questions and more tools to work with as we will see next.
Introducing more tools
Let’s introduce another tool : currency_conversion and run the same query as above.
currency_conversion = Tool(
name="Currency conversion",
func=convert_currency_to_usd,
description="useful for converting currency into USD. Input should be an amount.",
)Here are some helper functions this tool needs:
def value_to_float(x):
if type(x) == float or type(x) == int:
return x
x = x.upper()
if "MILLION" in x:
if len(x) > 1:
return float(x.replace("MILLION", "")) * 1000000
return 1000000.0
if "BILLION" in x:
return float(x.replace("BILLION", "")) * 1000000000
return 0.0
def convert_currency_to_usd(amount: str) -> int:
"Converts currency into USD"
if "¥" in amount:
exclude_symbol = amount.replace("¥", "")
amount_in_numbers = value_to_float(exclude_symbol)
return amount_in_numbers / 147.72 #harcoded the exchange rate here for simplicity
if "$" in amount:
exclude_symbol = amount.replace("$", "")
return value_to_float(exclude_symbol)Let’s re-run the earlier query:
tools = [tool_search, currency_conversion]
react_agent = create_react_agent(llm, tools, prompt)
agent_executor = AgentExecutor(
agent=react_agent, tools=tools, verbose=True, handle_parsing_errors=True
)
query = "Is the budget for Japan different than United States?"
agent_executor.invoke({"input": query})
## OUTPUT ##
> Entering new AgentExecutor chain...
I should check the HR policy to see if there is any information about budget differences between Japan and the United States.
Action: search_hr_policy
Action Input: "budget differences Japan United States"[Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Health and Safety:**\nWorkplace safety is a shared responsibility. Emergency procedures are clearly posted throughout our office buildings, and an annual budget of $10,000, €5,000, and ¥1 million is allocated for safety drills and equipment maintenance in the U.S., Germany, and Japan, respectively.\n\n**Communication:**\nImportant updates are conveyed through company-wide emails and team meetings. An annual budget of $500,000, €250,000, and ¥25 million is allocated for communication initiatives, including employee engagement events in the U.S., Germany, and Japan, respectively.\n\nThis policy undergoes an annual review to ensure relevance and compliance. Welcome to GlobalCorp, where our commitment to a diverse, inclusive, and respectful workplace is the foundation of our success.\n\n1.\tRecruitment and Selection\n1.\tIntroduction', metadata={'source': '../data/globalcorp_hr_policy.txt'})]I found information about budget differences between Japan and the United States in the HR policy.
Action: Currency conversion
Action Input: ¥50 million338478.20200379094The budget for Japan is equivalent to $338,478.20 USD.
Final Answer: The budget for Japan is different than the United States.
> Finished chain.
{'input': 'Is the budget for Japan different than United States?',
'output': 'The budget for Japan is different than the United States.'}You may be wondering what’s the big deal about this answer. Even without this tool, the previous answer we received was correct. However, what’s noteworthy here is its capability to perform currency conversions to USD before arriving at the ultimate conclusion that the budgets are indeed different. This helps build trust in the responses. Without this tool, I am willing to bet that if the HR policy stated the budget for Japan and the US as ¥741 million and $5 million, respectively, the LLM would respond that they have different budgets even though after conversion (as per today’s rate) they should be same.
A couple of observations:
- Given that the tools and their description get appended to the prompt, the model knows it has access to these tools if needed. I believe it helps them utilize as many of these tools as they deem necessary during answering a query. Hence, the decision to use the currency conversion tool in the second action.
Let’s update the query to get the actual difference in figures.
query = "Calculate the difference in company budget for Japan and United States?"
agent_executor.invoke({"input": query})
## OUTPUT ##
> Entering new AgentExecutor chain...
I need to find the company budget for Japan and the United States and then calculate the difference.
Action: search_hr_policy
Action Input: "company budget Japan"[Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Leave Policies - Germany:**\nIn Germany, generous leave policies offer 30 days of paid vacation and 20 days of paid sick leave annually. An annual budget of €1 million is allocated for leave-related expenses.\n\n**Leave Policies - Japan:**\nIn Japan, employees enjoy 20 days of paid vacation and 15 days of paid sick leave per year. An annual budget of ¥100 million is allocated for leave-related expenses.\n\n**Performance Management:**\nPerformance reviews are conducted annually, with regular feedback provided to support professional development. GlobalCorp encourages continuous learning and allocates an annual budget of $5,000 per employee for training and development opportunities.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Health and Safety:**\nWorkplace safety is a shared responsibility. Emergency procedures are clearly posted throughout our office buildings, and an annual budget of $10,000, €5,000, and ¥1 million is allocated for safety drills and equipment maintenance in the U.S., Germany, and Japan, respectively.\n\n**Communication:**\nImportant updates are conveyed through company-wide emails and team meetings. An annual budget of $500,000, €250,000, and ¥25 million is allocated for communication initiatives, including employee engagement events in the U.S., Germany, and Japan, respectively.\n\nThis policy undergoes an annual review to ensure relevance and compliance. Welcome to GlobalCorp, where our commitment to a diverse, inclusive, and respectful workplace is the foundation of our success.\n\n1.\tRecruitment and Selection\n1.\tIntroduction', metadata={'source': '../data/globalcorp_hr_policy.txt'})]I need to find the company budget for Japan and the United States and then calculate the difference.
Action: search_hr_policy
Action Input: "company budget United States"[Document(page_content='**Grievance and Disciplinary Procedures:**\nOur grievance and disciplinary procedures are outlined on the company intranet. Termination conditions may include gross misconduct or repeated policy violations. In such cases, a disciplinary process will be followed, including a three-strike system, before termination. Employees leaving GlobalCorp should follow the exit process detailed in the employee handbook.\n\n**Confidentiality and Data Security:**\nEmployees are expected to maintain confidentiality, and strict data security measures are in place. An annual budget of $1 million is allocated for cybersecurity training and awareness programs in the U.S., while budgets for Germany and Japan are €500,000 and ¥50 million, respectively.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Compensation and Benefits - United States:**\nIn the United States, employees enjoy health insurance, dental coverage, and a 401(k) retirement plan. An annual budget of $1 million is allocated for employee benefits.\n\n**Compensation and Benefits - Germany:**\nIn Germany, benefits include health insurance, a pension plan, and an additional 5 days of vacation annually. An annual budget of €500,000 is allocated for employee benefits.\n\n**Compensation and Benefits - Japan:**\nIn Japan, employees receive health insurance, a commuting allowance, and wellness benefits. An annual budget of ¥50 million is allocated for employee benefits.\n\n**Leave Policies - United States:**\nIn the United States, employees have 15 days of paid vacation and 10 days of paid sick leave annually. An annual budget of $500,000 is allocated for leave-related expenses.', metadata={'source': '../data/globalcorp_hr_policy.txt'}), Document(page_content='**Health and Safety:**\nWorkplace safety is a shared responsibility. Emergency procedures are clearly posted throughout our office buildings, and an annual budget of $10,000, €5,000, and ¥1 million is allocated for safety drills and equipment maintenance in the U.S., Germany, and Japan, respectively.\n\n**Communication:**\nImportant updates are conveyed through company-wide emails and team meetings. An annual budget of $500,000, €250,000, and ¥25 million is allocated for communication initiatives, including employee engagement events in the U.S., Germany, and Japan, respectively.\n\nThis policy undergoes an annual review to ensure relevance and compliance. Welcome to GlobalCorp, where our commitment to a diverse, inclusive, and respectful workplace is the foundation of our success.\n\n1.\tRecruitment and Selection\n1.\tIntroduction', metadata={'source': '../data/globalcorp_hr_policy.txt'})]I have found the company budget for Japan and the United States. Now I can calculate the difference.
Action: Currency conversion
Action Input: ¥50 million338478.20200379094I have converted the budget for Japan into USD.
Action: Currency conversion
Action Input: $1 million1000000.0I have converted the budget for the United States into USD.
Thought: I have the budgets for Japan and the United States in USD. Now I can calculate the difference.
Final Answer: The difference in company budget for Japan and the United States is $661,521.80.
> Finished chain.
{'input': 'Calculate the difference in company budget for Japan and United States?',
'output': 'The difference in company budget for Japan and the United States is $661,521.80.'}The LLM has done a great job at handling the subtraction (although I remain cautious about relying on LLMs for any type of calculations.). If we want to make it even more robust, we can add another tool, say calculator_subtract for calculating the difference between two numbers. As I mentioned before, ReAct agents cannot handle multi-input tools, and doing so would raise an error. This is where Open AI Tools agents come in picture.
Open AI Tools Agent
Let’s create a new tool — perc_diff()that takes two numbers as inputs and calculates the difference in percentage between these two numbers.
class Metrics(BaseModel):
num1: float = Field(description="Value 1")
num2: float = Field(description="Value 2")
@tool(args_schema=Metrics)
def perc_diff(num1: float, num2: float) -> float:
"""Calculates the percentage difference between two numbers"""
return (np.abs(num1 - num2) / ((num1+num2)/2)) * 100Note: Another way to initialize the same tool (giving more control over the setup)
from langchain.tools import BaseTool, StructuredTool
class Metrics(BaseModel):
num1: float = Field(description="Value 1")
num2: float = Field(description="Value 2")
def perc_diff(num1: float, num2: float) -> float:
"""Calculates the percentage difference between two numbers"""
return (np.abs(num1 - num2) / ((num1+num2)/2)) * 100
percentage_difference = StructuredTool.from_function(
func=perc_diff,
name="PercentageDifference", # make sure there are no spaces as OAI Tools agent will throw an error
description="calculates percentage difference between 2 numbers",
args_schema=Metrics,
return_direct=False,
)Finally, with all the pieces in place, let’s use create_openai_tools_agent
tools = [tool_search, currency_conversion, perc_diff]
prompt_oai_tools = hub.pull("hwchase17/openai-tools-agent")
oaitools_agent = create_openai_tools_agent(llm, tools, prompt_oai_tools)
oaitools_agent_executor = AgentExecutor(
agent=oaitools_agent, tools=tools, verbose=True, handle_parsing_errors=True
)
query = "As per the HR policy, compare the budgets for Japan and US."
oaitools_agent_executor.invoke({"input": query})
## OUTPUT ##
> Entering new AgentExecutor chain...
Invoking: `search_hr_policy` with `{'query': 'employee benefits budget Japan'}`
[Document(page_content='**Compensation and Benefits - Japan:**\nIn Japan,....]
Invoking: `search_hr_policy` with `{'query': 'employee benefits budget US'}`
[Document(page_content='**Compensation and Benefits - United States:**\nIn the United States,....]
Invoking: `Currency_conversion` with `JPY 50000000`
338478.20200379094
Invoking: `Currency_conversion` with `USD 1000000`
USD 1000000
Invoking: `percentage_difference` with `{'metric1': 338478.20200379094, 'metric2': 1000000}`
98.85The percentage difference in budgets for employee benefits between Japan and the US is approximately 98.85%, with the US budget being higher.
> Finished chain.
{'input': 'As per the HR policy, what is the percentage difference in budgets for employee benefits in Japan vs US.',
'output': 'The percentage difference in budgets for employee benefits between Japan and the US is approximately 98.85%, with the US budget being higher.'}Note: Upon running this line of code, you may notice errors like this i.e.Unrecognized request argument supplied: tools. It means that under the hood when the API call is made to the llm, it does not recognize the tools parameter. Given that only the newer version of the APIs recognize this parameter, it must mean you are using an older version of the model. You can fix this by:
- if you are using Azure Open AI services — deploy one of the newer models (see image below) and update the
deployment_namein your codebase i.e.llm=AzureChatOpenAI(deployment_name=..., )
- if you are using Open AI’s API directly — check that the model is a newer one from this list.
To test this fixed the issue, here’s a comparison before and after the version update (complete code snippet can be found here):
# Before
# Model name: gpt-35-turbo
# Model version: 0301
response = llm(messages=[message], tools=tools)
## OUTPUT ##
openai.error.InvalidRequestError: Unrecognized request argument supplied: tools# After
# Model name: gpt-35-turbo
# Model version: 0613
response = llm(messages=[message], tools=tools)
## OUTPUT ##
content='' additional_kwargs={'tool_calls': [<OpenAIObject id=call_8lvikb3ZqflGrr2xGgPGWXoJ at 0x10aa67270> JSON: {
"id": "call_8lvikb3ZqflGrr2xGgPGWXoJ",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\n \"location\": \"San Francisco\"\n}"
}
}]}Conclusion
Having taken a deep dive into the inner workings of the ReAct agent, I hope you feel more confident implementing it for your projects. This article just scratched the surface, there is so much more to cover. For instance, how to add memory to these QnA systems so you can use them in a chat-like manner.
As always if there’s an easier way to do/explain some of the things mentioned in this article, do let me know. In general, refrain from unsolicited destructive/trash/hostile comments!
Until next time ✨
