The world’s leading publication for data science, AI, and ML professionals.

How to Use OpenAI’s Function Calling

An Overview of Function Calling and Its Implications for Building LLM Apps

Image by Author: Generated with Midjourney
Image by Author: Generated with Midjourney

Structuring the Unstructured

Function calling is an innovation from OpenAI that has expanded the possibilities when it comes to app development around large language models.

However, I have found that it remains misunderstood by some. In this article, I aim to clarify function calling in the time it takes you to make a cup of coffee.

If you have aspirations to build LLM apps, integrate LLMs into your business, or simply expand your knowledge in this area, then this article is for you.


What’s So Great About Function Calling?

Function calling allows us to develop natural language interfaces atop our existing APIs. If this sounds confusing to you, don’t worry – the details will become clearer as you read on.

So, what does a natural language API look like? I believe it’s best to demonstrate this diagrammatically. Here’s an example app that uses function calling to enable users to find flights.

Image by Author: Diagrammatic representation of function calling
Image by Author: Diagrammatic representation of function calling

To implement this type of app without function calling, you would need to prompt the user to input the flight information using menus, selection boxes, and the like.

Function calling also facilitates the possibility of users making requests by voice. All you would need is an additional transcription service, and voilà, you have an AI personal assistant.

Hopefully, you should now be forming a clearer picture of function calling and its purpose. Let’s cement this newfound knowledge with some technical walkthroughs.


Function Calling with One Function

Let’s examine a simple use case atop a weather API.

The most basic use case involves function calling with one function. To illustrate this, I have modelled an API that provides a temperature forecast – see here.

Please take a moment to study this temperature API; note that it has two arguments: location and date.

Now, suppose we wanted to build an app based on this API. The first step we might take is to write a script that utilises a function call for our user queries. It would look something like this:

Let’s discuss the function calling. We define our functions as a list of dictionaries. The function allows us to extract arguments from a user query, which we can then use to feed our API. In our case, the arguments we wish to extract are "location" and "date".

We state the name of our function, in this case, "get_temperature", and provide a description of what it does.

Next, we need to describe the arguments of the function, which is crucial. What we receive back from the function call is influenced by how we do this. Take a look at how I have outlined the descriptions and types for the two arguments "location" and "date".

The descriptions are significant because function calling is powered by OpenAI’s large language models, which interpret the descriptions and respond accordingly.

Next, we set up our OpenAI model. The setup resembles a standard text generation use case with two key differences – the "functions" parameter and the "function_call" parameter.

We pass the functions we defined in the dictionary to the "functions" parameter. For the "function_call" parameter, we set it to "auto", allowing the large language model to decide when to use a function or not use one at all.

So, let’s see what happens when we run a query through this function call.

Here’s our query:

query = "What’s the temperature forecast in Berlin on the 1st of September 2023?"

And here’s the response:

<OpenAIObject at 0x1fd8de69ea0> JSON: {
  "role": "assistant",
  "content": null,
  "function_call": {
    "name": "get_temperature",
    "arguments": "{n  "location": "Berlin",n  "date": "2023-09-01"n}"
  }
}

The first thing to note is that the function call has returned a structured response, a json object.

Secondly, the function call has determined from our query that we wish to use the "get_temperature" function, and it has parsed the arguments for that function (location and date) accordingly.

This is excellent as we can now send these arguments to the weather API we constructed and obtain a response about the temperature.

# Parse the location and date from our function call response 

def parse_function(response):
    parsed_output = response["function_call"]["arguments"]
    parsed_output = json.loads(parsed_output)
    return parsed_output

location = parse_function(response)['location']
date = parse_function(response)['date']

# Get our weather data

data = get_temperature(location, date)

And here’s the response we get back from our weather API:

'{"location": "Berlin", "date": "2023-09-01", "temerature": 9.72, "scale": "Farenheit"}'

Note: It’s important to understand that function calling does not execute the function. It merely parses out the arguments for your function from a natural language input.


Function Calling with Multiple Functions

We can even specify multiple functions to "call" from. This is particularly useful when we have a natural language interface sitting atop multiple APIs. Let’s illustrate this with an example.

To complement our weather API, I have modelled a news headline API, which generates a mock news headline based on a topic.

The topic itself will be requested from a natural language query, which we can parse using function calling.

We can add this to our existing function call. Here’s how it appears:

Because we have set the "function_call" parameter to "auto", the large language model can deduce the most suitable function to call based solely on the query. Let’s see how this works for two different queries.

query_1 = "What’s the temperature forecast in NYC at the end of September 2023?"

Response from query 1:

<OpenAIObject at 0x1fd8de6b590> JSON: {
  "role": "assistant",
  "content": null,
  "function_call": {
    "name": "get_temperature",
    "arguments": "{n  "location": "NYC",n  "date": "2023-09-30"n}"
  }
}

Now let’s run the multi-function function call on a different query.

query_2 = "Give me the latest headline news on Berlin"

Response from query 2:

<OpenAIObject at 0x1fd8de67e00> JSON: {
  "role": "assistant",
  "content": null,
  "function_call": {
    "name": "get_current_news",
    "arguments": "{n  "topic": "Berlin"n}"
  }
}

In both cases, the function call has correctly selected which function to get the arguments for based on the queries.


What If We Don’t Use It?

At this point, you might be asking yourself: "This is great, but can’t I achieve the same result with the chat completion API and a prompt? Why do we need function calling at all?"

Well, the short answer is structure. While the chat completion API is trained to respond to a user query, you can prompt it to perform a "function call", but the output you get may not be structured, making it difficult to parse the arguments.

Consider this for example:

Here we have skipped function calling altogether and simply asked the chat completion API to respond with our arguments with some prompting.

And the response:

'The news topic required is "Berlin".'

Although the answer is correct, it’s more difficult to parse as an argument because it is unstructured.

There is also the issue of consistency. You can attempt to enforce some structure through prompting, but the responses may not always be structured as you desire.

Function calling guarantees a structured response every time, making it a more reliable method of parsing arguments to feed a downstream function.


Conclusion

Function calling opens the door to the development of a host of new LLM-based applications. We can now consistently generate structured outputs from unstructured data.

If you wish to explore how you can build your first LLM app, read this:

How to Build an LLM Application

Thanks for reading.

The full notebook, is available via my GitHub repo. There is also a live video tutorial on YouTube.


If you’re keen to enhance your skills in Artificial Intelligence, join the waiting list for my course, where I will guide you through the process of developing large language model powered applications.

If you’re seeking AI-transformation for your business, book a discovery call today.

Brainqub3 | AI software development

For more insights on artificial intelligence, Data Science, and large language models you can subscribe to the YouTube channel.


Related Articles