Building a Smart Travel Itinerary Suggester with LangChain, Google Maps API, and Gradio (Part 2)
Learn how to build an application that might inspire your next road trip
This article is part 2 of a three part series where we build a travel itinerary suggester application using OpenAI and Google APIs and display it in a simple UI generated with gradio. In this part, we discuss how to use the Google Maps API and folium to generate an interactive route map from a list of waypoints. Just want see the code? Find it here.
1. Recap of part 1
In the first of this three part series, we used LangChain and prompt engineering to build a system that makes sequential calls to an LLM API — either Google’s PaLM or OpenAI’s ChatGPT that convert a user’s query into a travel itinerary and a nicely parsed list of addresses. Now it’s time to see how we can take that list of addresses and convert it into a travel route with directions plotted on a map. To do this, we will primarily be making use of the Google Maps API via the googlemaps package. We’ll also use folium for plotting. Let’s get started!
2. Getting ready to make API calls
In order to make an API key for Google Maps, you’ll first need to make an account with Google Cloud. They have a 90-day free trial period, after which you’ll pay for API services you use in a similar way to what you’d do with OpenAI. Once that’s complete, you can make a project (mine is called LLMMapper) and navigate the Google Maps Platform section of the Google Cloud site. From there, you should be able to access the “Keys & Credentials” menu to generate an API key. You should also check out the “APIs & Services” menu to explore the many services that the Google Maps Platform provides. For this project we’ll just be using the Directions and Geocoding services. We’ll be geocoding each of our waypoints and then finding directions between them.
Now, the Google Maps API key can be added to the .env file that we set up earlier
OPENAI_API_KEY = {your open ai key}
GOOGLE_PALM_API_KEY = {your google palm api key}
GOOGLE_MAPS_API_KEY = {your google maps api key here}To test if this works, load the secrets from the.env using the method described in part 1. We can then attempt a geocoding call as follows
import googlemaps
def convert_to_coords(input_address):
return self.gmaps.geocode(input_address)
secrets = load_secets()
gmaps = googlemaps.Client(key=secrets["GOOGLE_MAPS_API_KEY"])
example_coords = convert_to_coords("The Washington Moment, DC")Google Maps is able to match the supplied string with the address and details of an actual place, and should return a list like this
[{'address_components': [{'long_name': '2',
'short_name': '2',
'types': ['street_number']},
{'long_name': '15th Street Northwest',
'short_name': '15th St NW',
'types': ['route']},
{'long_name': 'Washington',
'short_name': 'Washington',
'types': ['locality', 'political']},
{'long_name': 'District of Columbia',
'short_name': 'DC',
'types': ['administrative_area_level_1', 'political']},
{'long_name': 'United States',
'short_name': 'US',
'types': ['country', 'political']},
{'long_name': '20024', 'short_name': '20024', 'types': ['postal_code']}],
'formatted_address': '2 15th St NW, Washington, DC 20024, USA',
'geometry': {'location': {'lat': 38.8894838, 'lng': -77.0352791},
'location_type': 'ROOFTOP',
'viewport': {'northeast': {'lat': 38.89080313029149,
'lng': -77.0338224697085},
'southwest': {'lat': 38.8881051697085, 'lng': -77.0365204302915}}},
'partial_match': True,
'place_id': 'ChIJfy4MvqG3t4kRuL_QjoJGc-k',
'plus_code': {'compound_code': 'VXQ7+QV Washington, DC',
'global_code': '87C4VXQ7+QV'},
'types': ['establishment',
'landmark',
'point_of_interest',
'tourist_attraction']}]This is very powerful! Although the request is somewhat vague, the Google Maps service has correctly matched it to an exact address with coordinates and various other locale information that might be useful to a developer depending on the application. We will only need to make use of the formatted_address and place_idfields here.
3. Building the route
Geocoding is important for our travel mapping application because the geocoding API appears more adept at handling vague or partially compete addresses than the directions API. There is no guarantee that the addresses coming from the LLM calls will be contain enough information for the directions API to give a good response, so doing this geocoding step first decreases the likelihood of errors.
Lets first call the geocoder on the start point, end point and intermadiate waypoints list and store the results in a dictionary
def build_mapping_dict(start, end, waypoints):
mapping_dict = {}
mapping_dict["start"] = self.convert_to_coords(start)[0]
mapping_dict["end"] = self.convert_to_coords(end)[0]
if waypoints:
for i, waypoint in enumerate(waypoints):
mapping_dict["waypoint_{}".format(i)] = convert_to_coords(
waypoint
)[0
return mapping_dictNow, we can make use of the directions API to get the route from start to end that includes the waypoints
def build_directions_and_route(
mapping_dict, start_time=None, transit_type=None, verbose=True
):
if not start_time:
start_time = datetime.now()
if not transit_type:
transit_type = "driving"
# later we replace this with place_id, which is more efficient
waypoints = [
mapping_dict[x]["formatted_address"]
for x in mapping_dict.keys()
if "waypoint" in x
]
start = mapping_dict["start"]["formatted_address"]
end = mapping_dict["end"]["formatted_address"]
directions_result = gmaps.directions(
start,
end,
waypoints=waypoints,
mode=transit_type,
units="metric",
optimize_waypoints=True,
traffic_model="best_guess",
departure_time=start_time,
)
return directions_resultFull documentation for the directions api is here, and there are many different options that can be specified. Notice that we specific the start and end of route along with the list of waypoints, and choose optimize_waypoints=True so that Google Maps knows that the order of the waypoints can be changed in order to reduce the total travel time. We can also specify transit type too, which defaults to driving unless otherwise set. Recall that in part 1 we asked the LLM to return transit type along with its itinerary suggestion, so in theory we could make use of that here too.
The dictionary returned from the directions API call has the following keys
['bounds',
'copyrights',
'legs',
'overview_polyline',
'summary',
'warnings',
'waypoint_order'
]Of this information, legs and overview_polyline will be the most useful to us. legs is a list of route segments, each element of which looks like this
['distance',
'duration',
'end_address',
'end_location',
'start_address',
'start_location',
'steps',
'traffic_speed_entry',
'via_waypoint'
]Each leg is further segmented into steps, which is the collection of turn-by-turn instructions and their associated route segments. This a list of dictionaries with the following keys
['distance',
'duration',
'end_location',
'html_instructions',
'polyline',
'start_location',
'travel_mode'
]The polylinekeys are where the actual route information is stored. Each polyline is an encoded representation of a list of coordinates, which Google Maps generates as a means of compressing a long list of latitude and longitude values into a string. They are encoded strings that look like
“e|peFt_ejVjwHalBzaHqrAxeE~oBplBdyCzpDif@njJwaJvcHijJ~cIabHfiFyqMvkFooHhtE}mMxwJgqK”
You can read more about this here, but thankfully we can use the decode_polyline utility to convert them back into coordinates. For example
from googlemaps.convert import decode_polyline
overall_route = decode_polyline(
directions_result[0]["overview_polyline"]["points"]
)
route_coords = [(float(p["lat"]),float(p["lng"])) for p in overall_route]This will give a list of latitude and longitude points along the route.
This is all we need to know to plot a simple map showing the waypoints on a route and the correct driving paths connecting them. We can use the overview_polyline as a start, although we will see later on that this can cause resolution issues at high zoom levels of the map.
Lets assume we started with the following query:
“I want to do 5 day road trip from San Francisco to Las Vegas. I want to visit pretty coastal towns along HW1 and then mountain views in southern California”
Our LLM calls extracted a dictionary of waypoints and we ran build_mapping_dict and build_directions_and_route to get our directions result from Google Maps
We can first extract the waypoints like this
marker_points = []
nlegs = len(directions_result[0]["legs"])
for i, leg in enumerate(directions_result[0]["legs"]):
start, start_address = leg["start_location"], leg["start_address"]
end, end_address = leg["end_location"], leg["end_address"]
start_loc = (float(start["lat"]),float(start["lng"]))
end_loc = (float(end["lat"]),float(end["lng"]))
marker_points.append((start_loc,start_address))
if i == nlegs-1:
marker_points.append((end_loc,end_address))Now, using folium and branca, we can plot a nice interactive map which should appear in Colab or Jupyter Notebook
import folium
from branca.element import Figure
figure = Figure(height=500, width=1000)
# decode the route
overall_route = decode_polyline(
directions_result[0]["overview_polyline"]["points"]
)
route_coords = [(float(p["lat"]),float(p["lng"])) for p in overall_route]
# set the map center to be at the start location of the route
map_start_loc = [overall_route[0]["lat"],overall_route[0]["lng"]]
map = folium.Map(
location=map_start_loc,
tiles="Stamen Terrain",
zoom_start=9
)
figure.add_child(map)
# Add the waypoints as red markers
for location, address in marker_points:
folium.Marker(
location=location,
popup=address,
tooltip="<strong>Click for address</strong>",
icon=folium.Icon(color="red", icon="info-sign"),
).add_to(map)
# Add the route as a blue line
f_group = folium.FeatureGroup("Route overview")
folium.vector_layers.PolyLine(
route_coords,
popup="<b>Overall route</b>",
tooltip="This is a tooltip where we can add distance and duration",
color="blue",
weight=2,
).add_to(f_group)
f_group.add_to(map)When this code is run, Folium will generate an interactive map that we can explore and click into each of the waypoints.
4. Refining the route
The approach described above where we make a single call to the Google Maps directions API with a list of waypoints and then plot the overview_polyline works great as a POC, but there a few issues:
- Instead of
formatted_address, it is more efficient to useplace_idwhen specifying the start, end and waypoint names in the call to Google Maps. Fortunately we getplace_idin the result of our geocoding calls, so we should use it. - The number of waypoints that can be requested in a single API call is limited to 25 (see https://developers.google.com/maps/documentation/directions/get-directions for details). If we have have more than 25 stops in our itinerary from the LLM, we need to make more calls to Google Maps and then merge the responses
overview_polylinehas limited resolution when we zoom in, likely since the number of points along it are optimized for a large-scale map view. This is not a major issue for a POC, but it would be nice to have some more control over the route resolution so that it looks good even at high zoom levels. The directions API is giving us much more granular polylines in the route segments that it provides, so we can make use of these.- On the map, it would be nice to split the route into separate legs and allow the user to see the distance and travel times associated with each one. Again, Google Maps is providing us with that information so we should make use of it.
Issue 1 can easily be solved just by modifying build_directions_and_route to make use of place_id from the mapping_dict rather than formatted_address . Issue 2 is a little more involved and requires breaking our initial waypoints down into chunks of some maximum length, creating a start, end and sub-list of waypoints from each and then running build_mapping_dict followed by build_directions_and_route on those. The results can then be concatenated at the end.
Issues 3 and 4 can be solved by using the individual step polylines for each leg of the route returned by Google Maps. We just need to loop though these two levels, decode the relevant polylines and then construct a new dictionary. This also enables us to extract the distance and duration values, which get assigned to each decoded leg and then used for plotting.
def get_route(directions_result):
waypoints = {}
for leg_number, leg in enumerate(directions_result[0]["legs"]):
leg_route = {}
distance, duration = leg["distance"]["text"], leg["duration"]["text"]
leg_route["distance"] = distance
leg_route["duration"] = duration
leg_route_points = []
for step in leg["steps"]:
decoded_points = decode_polyline(step["polyline"]["points"])
for p in decoded_points:
leg_route_points.append(f'{p["lat"]},{p["lng"]}')
leg_route["route"] = leg_route_points
waypoints[leg_number] = leg_route
return waypointsThe trouble now is that the leg_route_pointslist can become very long, and when we come to plot this on the map it can cause folium to crash or run very slowly. The solution is to sample the points along the route so that there are enough to allow a good visualization but not too many that the map has trouble loading.
A simple and safe way to do this is calculate the number of points that the total route should contain (say 5000 points) then determine what fraction should belong to each leg of the route and then evenly sample the corresponding number of points from each leg. Note that we need to make sure that each leg contains at least one point for it to get included on the map.
The following function will do this sampling, taking in a dictionary of waypoints output from the get_route function above.
def sample_route_with_legs(route, distance_per_point_in_km=0.25)):
all_distances = sum([float(route[i]["distance"].split(" ")[0]) for i in route])
# Total points in the sample
npoints = int(np.ceil(all_distances / distance_per_point_in_km))
# Total points per leg
points_per_leg = [len(v["route"]) for k, v in route.items()]
total_points = sum(points_per_leg)
# get number of total points that need to be represented on each leg
number_per_leg = [
max(1, np.round(npoints * (x / total_points), 0)) for x in points_per_leg
]
sampled_points = {}
for leg_id, route_info in route.items():
total_points = int(points_per_leg[leg_id])
total_sampled_points = int(number_per_leg[leg_id])
step_size = int(max(total_points // total_sampled_points, 1.0))
route_sampled = [
route_info["route"][idx] for idx in range(0, total_points, step_size)
]
distance = route_info["distance"]
duration = route_info["duration"]
sampled_points[leg_id] = {
"route": [
(float(x.split(",")[0]), float(x.split(",")[1]))
for x in route_sampled
],
"duration": duration,
"distance": distance,
}
return sampled_pointsHere we specify the spacing of points that we want — one per 250m — and then choose the number of points accordingly. We might also consider implementing a way to estimate the desired point spacing from the length of the route, but this method appears to work reasonably well for a first pass, giving acceptable resolution at moderately high levels of zoom on the map.
Now that we’ve split the route up into legs with a reasonable number of sample points, we can proceed to plot them on the map and label each leg with the following code
for leg_id, route_points in sampled_points.items():
leg_distance = route_points["distance"]
leg_duration = route_points["duration"]
f_group = folium.FeatureGroup("Leg {}".format(leg_id))
folium.vector_layers.PolyLine(
route_points["route"],
popup="<b>Route segment {}</b>".format(leg_id),
tooltip="Distance: {}, Duration: {}".format(leg_distance, leg_duration),
color="blue",
weight=2,
).add_to(f_group)
# assumes the map has already been generated
f_group.add_to(map)
5. Putting it all together
In the codebase, all the methodology mentioned above is packaged in two classes. The first is RouteFinder , which takes in the stuctured output of Agent (see part 1) and generates the sampled route. The second is RouteMapper , which takes the sampled route and plots a folium map, which can be saved as html.
Since we almost always want to generate a map when we ask for a route, the RouteFinder's generate_route method handles both of these tasks
class RouteFinder:
MAX_WAYPOINTS_API_CALL = 25
def __init__(self, google_maps_api_key):
self.logger = logging.getLogger(__name__)
self.logger.setLevel(logging.INFO)
self.mapper = RouteMapper()
self.gmaps = googlemaps.Client(key=google_maps_api_key)
def generate_route(self, list_of_places, itinerary, include_map=True):
self.logger.info("# " * 20)
self.logger.info("PROPOSED ITINERARY")
self.logger.info("# " * 20)
self.logger.info(itinerary)
t1 = time.time()
directions, sampled_route, mapping_dict = self.build_route_segments(
list_of_places
)
t2 = time.time()
self.logger.info("Time to build route : {}".format((round(t2 - t1, 2))))
if include_map:
t1 = time.time()
self.mapper.add_list_of_places(list_of_places)
self.mapper.generate_route_map(directions, sampled_route)
t2 = time.time()
self.logger.info("Time to generate map : {}".format((round(t2 - t1, 2))))
return directions, sampled_route, mapping_dictRecall that in part 1 we constructed a class called Agent , which handled the LLM calls. Now that we also have RouteFinder , we can put them together in a base class for the entire travel mapper project
class TravelMapperBase(object):
def __init__(
self, openai_api_key, google_palm_api_key, google_maps_key, verbose=False
):
self.travel_agent = Agent(
open_ai_api_key=openai_api_key,
google_palm_api_key=google_palm_api_key,
debug=verbose,
)
self.route_finder = RouteFinder(google_maps_api_key=google_maps_key)
def parse(self, query, make_map=True):
itinerary, list_of_places, validation = self.travel_agent.suggest_travel(query)
directions, sampled_route, mapping_dict = self.route_finder.generate_route(
list_of_places=list_of_places, itinerary=itinerary, include_map=make_map
)This can be run on a query as follows, which is the example given in the test_without_gradio script
from travel_mapper.TravelMapper import load_secrets, assert_secrets
from travel_mapper.TravelMapper import TravelMapperBase
def test(query=None):
secrets = load_secrets()
assert_secrets(secrets)
if not query:
query = """
I want to do 2 week trip from Berkeley CA to New York City.
I want to visit national parks and cities with good food.
I want use a rental car and drive for no more than 5 hours on any given day.
"""
mapper = TravelMapperBase(
openai_api_key=secrets["OPENAI_API_KEY"],
google_maps_key=secrets["GOOGLE_MAPS_API_KEY"],
google_palm_api_key=secrets["GOOGLE_PALM_API_KEY"],
)
mapper.parse(query, make_map=True)In terms of the route and map generation we’re now done! But how can we package all this code into a nice UI thats easy to experiment with? That will be covered in the third and final part of this series.
Thanks for reading! Please feel free to explore the full codebase here https://github.com/rmartinshort/travel_mapper. Any suggestions for improvement or extensions to the functionality would be much appreciated!
