Grounding with Google Maps in Vertex AI

This guide shows how to use Grounding with Google Maps in Vertex AI to enhance your generative AI applications by providing geospatial context.

This document covers the following topics:

The following diagram summarizes the overall workflow:

Overview

Grounding with Google Maps in Vertex AI is a Preview service that connects Gemini models with geospatial data from Google Maps. Google Maps has access to information on over 250 million places, including businesses, landmarks, and points of interest. You can use this data to ground your model's responses, which enables your AI applications and agents to provide local data and geospatial context.

For technical support with Grounding with Google Maps, email [email protected].

Uses of Grounding with Google Maps

You can use Grounding with Google Maps for various applications, such as the following:

  • Chat-based agents
  • Summarization of place information
  • Content translation
  • Conversational assistants that can answer questions about nearby places, such as "Are there any parks nearby?"
  • Personalized descriptions of places, such as "Can you tell me more about the parks and any family-friendly restaurants that are within a walkable distance?"

This feature is useful for industries like real estate, travel, mobility, and social applications.

Supported models

The following models support grounding with Google Maps.

For more information about the Gemini models, see Gemini models.

Ground your model's responses

You can ground your model's responses by using the Vertex AI Studio to experiment with prompts or by using the REST API to integrate grounding into your applications.

Method Description Use Case
Vertex AI Studio A web-based UI for experimenting with prompts and grounding settings without writing code. Ideal for rapid prototyping, testing prompts, and exploring the feature's capabilities.
REST API A programmatic interface to integrate grounding directly into your applications. Best for production environments and building automated, scalable applications that leverage Grounding with Google Maps.

The following samples demonstrate how to use Grounding with Google Maps to ground your model's responses.

Console

To use Grounding with Google Maps with Vertex AI Studio, follow these steps:

  1. In the Google Cloud console, go to the Vertex AI Studio page.

    Go to Vertex AI Studio

  2. Click the Freeform tab.
  3. In the side panel, click the Ground model responses toggle.
  4. Click Customize, and set Google Maps as the source.
  5. Enter your prompt in the text box, and click Submit.

The model's responses are now grounded to Google Maps.

REST

Before using any of the request data, make the following replacements:

  • LOCATION: The region to process the request.
  • PROJECT_ID: Your project ID.
  • MODEL_ID: The model ID of the multimodal model.
  • TEXT: The text instructions to include in the prompt.
  • LATITUDE: The latitude of the location.
  • LONGITUDE: The longitude of the location.

HTTP method and URL:

POST https://LOCATION-aiplatform.googleapis.com/v1beta1/projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID:generateContent

Request JSON body:

 {   "contents": [{     "role": "user",     "parts": [{       "text": "TEXT"     }]   }],   "tools": [{     "googleMaps": {}   }],   "toolConfig": {     "retrievalConfig": {       "latLng": {         "latitude": LATITUDE,         "longitude": LONGITUDE       }     }   },   "model": "projects/PROJECT_ID/locations/LOCATION/publishers/google/models/MODEL_ID" } 

To send your request, expand one of these options:

You should receive a JSON response similar to the following:

 {   "candidates": [     {       "content": {         "role": "model",         "parts": [           {             "text": "\"The Italian Place\" in Alexandria, VA, is good for children and offers takeout. It has a rating of 4.2 stars based on 411 reviews."           }         ]       },       "finishReason": "STOP",       "groundingMetadata": {         "groundingChunks": [           {             "maps": {               "uri": "https://maps.google.com/?cid=9001322937822692826",               "title": "The Italian Place",               "text": "**About:**\n\n* **Type:** Italian Restaurant\n* **Address:** 621 Wythe St, Alexandria, VA 22314, USA\n* **Open Now:** Yes\n* **Rating:** 4.2 (411 reviews)\n* **Price Level:** Moderate\n* **Phone:** (571) 777-8981\n* **Summary:** Down-to-earth, counter-serve stop offering Italian sandwiches, coffee & market goods.\n* **Additional Summary:** Relaxed Italian eatery known for sandwiches and pizza along with gourmet food items and gelato.\n* **Offers Takeout:** Yes\n* **Offers Delivery:** Yes\n* **Offers Dine-in:** Yes\n* **Good for Children:** Yes\n* **Has Restroom:** Yes\n* **Outdoor Seating:** Yes\n* **Live Music:** No\n* **Menu for Children:** No\n* **Serves Dessert:** Yes\n* **Serves Coffee:** Yes\n* **Good for Watching Sports:** No\n* **Serves Lunch:** Yes\n* **Serves Dinner:** Yes\n* **Serves Beer:** Yes\n* **Serves Vegetarian Food:** Yes\n\n**Opening Hours (local time):**\n\n* Monday: 11:00 AM – 7:00 PM\n* Tuesday: 10:00 AM – 7:00 PM\n* Wednesday: 11:00 AM – 7:00 PM\n* Thursday: 11:00 AM – 7:00 PM\n* Friday: 11:00 AM – 7:00 PM\n* Saturday: 11:00 AM – 7:00 PM\n* Sunday: 12:00 – 7:00 PM\n\n**Parking options:**\n\n* **Free parking lot:** Yes\n* **Free street parking:** Yes\n* **Valet parking:** No\n\n**Accessibility:**\n\n* **Wheelchair accessible parking:** Yes\n* **Wheelchair accessible restroom:** Yes\n\n**Payment options:**\n\n* **Credit Card:** Yes\n* **Debit Card:** Yes\n* **Cash Only:** No\n* **NFC:** Yes\n\n**Distance & Travel Time:**\n\n* 384.6 kilometers\n* 4.2 hours",               "placeId": "places/ChIJOTRDf_qwt4kR2kV_WYUf63w"             }           },           ...         ],         "groundingSupports": [           {             "segment": {               "endIndex": 79,               "text": "\"The Italian Place\" in Alexandria, VA, is good for children and offers takeout."             },             "groundingChunkIndices": [               0             ]           },           {             "segment": {               "startIndex": 80,               "endIndex": 130,               "text": "It has a rating of 4.2 stars based on 411 reviews."             },             "groundingChunkIndices": [               0             ]           }         ],         "googleMapsWidgetContextToken": "widgetcontent/..."       }     }   ],   ... } 

Optional: Use the Google Maps contextual widget

The contextual widget is a Google Maps Pre-GA offering that you can use to integrate Grounding with Google Maps into your applications and create a conversational, LLM-powered chat experience. To render the widget, use the googleMapsWidgetContextToken that the Vertex AI API returns in its response.

The contextual widget serves different functions depending on your scenario:

  • It displays subjective user-generated content (UGC) when Google Maps prompting is used for answer generation.
  • It enriches results with map visualizations and data when Vertex AI generates only a text response.

For more information, see Maps grounding widget.

Render the Google Maps contextual widget

To render and use the Google Maps contextual widget, use the alpha version of the Google Maps JavaScript API on the page that displays the widget. For more information, see Load the Maps JavaScript API.

The following code samples demonstrate how to use a contextual widget.

HTML

1. Create a contextual widget.

<body>   <gmp-place-contextual id="widget"></gmp-place-contextual> </body> 

2. Update the context token.

Any response grounded with Google Maps includes a googleMapsWidgetContextToken. Use this token to render the contextual widget by setting the widget.contextToken property. Place the widget in close proximity to the generated response.

"googleMapsWidgetContextToken": "widgetcontent/AcBXPQdpWQWbap9H-OH8sEKmOXxmEKAYvff0tvthhneMQC3VrqWCjpnPBl4-Id98FGiA_S_t8aeAeJj0T6JkWFX56Bil8oBSR0W8JH3C_RSYLbTjxKdpxc9yNn6JcZTtolIRZon9xi6WpNGuSyjcIxWu2S0hwpasNOpUlWrG1RxVCB4WD1fsz_pwR236mG36lMxevXTQ_JnfdYNuQwQ4Lc3vn...<snip>... Ts5VJE_b3IC5eE_6wez0nh61r7foTUZpP7BXMwxR-7Wyfcj6x1v6mIWsFGr1o0p_HSAMNqWPg-aFVnkPLhAkOR6MaNZOfezTva-gxHlu7z_haFvYxcUE1qfNVQ", 
function updateWidget(contextToken) {   let widget = document.querySelector('#widget');   widget.contextToken = contextToken; } 

3. Optional: Customize the widget.

  • Specify the list layout. Valid values are compact (default) and vertical.

    <gmp-place-contextual id="widget">   <gmp-place-contextual-list-config layout="compact">   </gmp-place-contextual-list-config> </gmp-place-contextual> 
  • Change the map mode. Valid values are roadmap (2D map), hybrid (3D map), and none.

    <gmp-place-contextual id="widget">   <gmp-place-contextual-list-config map-mode="roadmap">   </gmp-place-contextual-list-config> </gmp-place-contextual> 

JavaScript

1. Create a contextual widget.

async function createWidget(contextToken) {   await google.maps.importLibrary('places');   let widgetContainer = document.querySelector('#wc');  // a div that contains the widget   const placeContextualElement = new       google.maps.places.PlaceContextualElement({ contextToken });   widgetContainer.appendChild(placeContextualElement); } 

2. Update the context token.

Any response grounded with Google Maps includes a googleMapsWidgetContextToken. Use this token to render the contextual widget by setting the widget.contextToken property. Place the widget in close proximity to the generated response.

"googleMapsWidgetContextToken": "widgetcontent/AcBXPQdpWQWbap9H-OH8sEKmOXxmEKAYvff0tvthhneMQC3VrqWCjpnPBl4-Id98FGiA_S_t8aeAeJj0T6JkWFX56Bil8oBSR0W8JH3C_RSYLbTjxKdpxc9yNn6JcZTtolIRZon9xi6WpNGuSyjcIxWu2S0hwpasNOpUlWrG1RxVCB4WD1fsz_pwR236mG36lMxevXTQ_JnfdYNuQwQ4Lc3vn...<snip>... Ts5VJE_b3IC5eE_6wez0nh61r7foTUZpP7BXMwxR-7Wyfcj6x1v6mIWsFGr1o0p_HSAMNqWPg-aFVnkPLhAkOR6MaNZOfezTva-gxHlu7z_haFvYxcUE1qfNVQ", 
      function updateWidget(contextToken) {         widget.contextToken = contextToken;       } 

3. Optional: Customize the widget.

  • Specify the list layout. Valid values are google.maps.places.PlaceContextualListLayout.COMPACT (default) and google.maps.places.PlaceContextualListLayout.VERTICAL.

    const widgetConfig = new google.maps.places.PlaceContextualListConfigElement({   layout: google.maps.places.PlaceContextualListLayout.COMPACT }); widget.appendChild(widgetConfig); 
  • Change the map mode. Valid values are google.maps.places.PlaceContextualListMapMode.ROADMAP (2D map), google.maps.places.PlaceContextualListMapMode.HYBRID (3D map), and google.maps.places.PlaceContextualListMapMode.NONE.

    const widgetConfig = new google.maps.places.PlaceContextualListConfigElement({   mapMode: google.maps.places.PlaceContextualListMapMode.ROADMAP }); widget.appendChild(widgetConfig); 

Source attribution requirements

When you present Vertex AI-generated content that references information from Grounding with Google Maps, you must specify the Google Maps sources that support the response.

This image shows Google Maps sources that were used to support the model's response.

Prompt with response showing sources

Inform the end-user about the use of grounding sources

You must inform your users that the LLM-generated content is supported by grounding sources. Place this information in close proximity to the content and make it viewable within one user interaction.

Display Google Maps source URLs

The API response returns Google Maps sources in the groundingMetadata field within groundingChunks and groundingSupports. Sources are returned for both Places and for supporting content like user reviews.

This code sample shows a Place source and a Place answer source in the response:

  "groundingChunks": [             {               "maps": {                 "uri": "{Link to Maps Content}",                 "title": "{Name of Maps Place}",                 "text": "{Maps content that was sent to the model for this place}"                 "placeId": "{Place ID}",                 "placeAnswerSources":                                   {                     "review": "",                     "authorAttribution": {                       "displayName": "",                       "photoUri": ""                     },                     "flagContentUri": "",                     "googleMapsUri": ""                   },               },                "flagContentUri": ""               }             }           }         ], 

For each source that the LLM references, you must generate a link preview that meets the following requirements:

  • Attribute to Google Maps: Follow the Text attribution guidelines.
  • Display the title: Use the Open Graph page title (og:title) or the format [Place Name] - Google Maps.
  • Link to the source: The link must navigate to the source material using the source URL.

If groundingChunks contain Place answer sources, you must also do the following:

  • Render review links: Use the URL from grounding_chunks.maps.placeAnswerSources.reviewSnippets.googleMapsUri.
  • Display the review title: Use the Open Graph Title or the format Google Review of [Place Name] by [Author Name]. Place Name is in grounding_chunks.maps.title and Author Name is in grounding_chunks.maps.placeAnswerSources.reviewSnippets.authorAttribution.displayName.

You can optionally enhance the link preview with additional content, such as:

  • Google Maps favicon (<link rel="icon" href="https://www.google.com/favicon.ico">) inserted before the Google Maps text attribution.
  • Description (og:description)
  • Photo (og:image)

These images show the minimum requirement, which is to show the Place links. You can collapse the view of the sources.

Prompt with response showing sources Prompt with response and sources collapsed

This image shows an optional enhancement, which includes a photo of the place.

Prompt with response and sources

Prohibited territories

Google Maps Platform restricts certain content and activities to maintain a safe and reliable platform. For a list of prohibited territories, see Google Maps Platform Prohibited Territories.

Place properties

This section lists the place properties that Grounding with Google Maps uses to generate responses. These properties determine the types of questions that Grounding with Google Maps can answer.

Sample place properties

The following is an alphabetized sample of properties about places that your model can use to generate responses.

  • Address
  • Curbside pickup
  • Debit card
  • Distance
  • Free parking lot
  • Live music
  • Menu for children
  • Opening hours
  • Payment options (such as cash or credit card)
  • Place answer
  • Pet friendly
  • Serves beer
  • Serves vegetarian food
  • Wheelchair accessible
  • Wifi

Place answers are responses from Grounding with Google Maps based on information from user reviews. If there is an issue with the content of a Place answer, you can report it to Google by using the link in the flagContentUri field inside the PlaceAnswerSources object in the API response.

Examples of using place properties

The following examples use place properties in questions about different types of places. Grounding with Google Maps uses these properties to understand your intent and provide relevant answers.

  • Plan a family dinner: Ask questions like, Is "The Italian Place" good for children, and do they offer takeout? What is their rating? This helps you determine if a restaurant is suitable for a family and offers convenient services.

  • Check accessibility for a friend: Ask questions like, I need a restaurant that has a wheelchair accessible entrance. This helps you determine if the location meets specific accessibility needs.

  • Find a location for a late-night snack: Ask questions like, Is "Burger Joint" open now? Do they serve dinner? What are their opening hours for Friday? This helps you find an open establishment serving a specific meal at a particular time.

  • Meet a client for coffee: Ask questions like, Does "Cafe Central" have Wifi? Do they serve coffee? What is their price level, and do they accept credit cards? This helps you assess the suitability of a cafe for a business meeting based on amenities, offerings, and payment options.

What's next

  • To learn more about how to ground Gemini models to your data, see Grounding with your data.
  • To learn more about responsible AI best practices and Vertex AI's safety filters, see Responsible AI.