Firebase AI Logic on Android: Gemini Integration Patterns Every Developer Should Know

Google published a deep-dive on their mixed-reality tour guide app last week — an Android XR project that combines the ARCore Geospatial API, Gemini 3.5 Flash, and Gemini 2.5 Flash TTS into a location-aware spatial guide. The XR angle is interesting, but what actually caught my attention was the Firebase AI Logic layer underneath it. That SDK is the production story for integrating Gemini into Android apps, and this app shows several patterns that aren't obvious from reading the documentation alone.

I've been watching how the Gemini integration story for Android has evolved, and Firebase AI Logic is now clearly the right path — not because of lock-in, but because it gives you AppCheck, per-request token counting, Google Maps Grounding (which solves a real problem), and a clean Kotlin API, all without wiring up the Gemini REST endpoints yourself. The tour guide app demonstrates three of those patterns in a single codebase in a way that's directly translatable to everyday Android apps.

1. Structured JSON Output the Right Way

Every Android developer who's tried LLM integration has hit this problem: you ask the model for structured data, it gives you something almost right, and now you're writing a regex to extract the JSON from a markdown code block. Firebase AI Logic's Schema API eliminates that entire class of bug.

Instead of asking for JSON in your prompt and hoping, you define a typed schema and set responseMimeType = "application/json" alongside a responseSchema. The model is then constrained to produce output that matches your schema — not markdown, not prose with embedded JSON, just the object you asked for.

In the tour guide app, the schema defines a tour object with nested stops:

val responseJsonSchema = Schema.obj(
  mapOf(
    "locationIntro" to Schema.string(),
    "tours" to Schema.array(
      Schema.obj(
        mapOf(
          "title" to Schema.string(),
          "description" to Schema.string(),
          "stops" to Schema.array(
            Schema.obj(
              mapOf(
                "name" to Schema.string(),
                "detailedName" to Schema.string(),
                "description" to Schema.string()
              )
            )
          )
        )
      )
    )
  )
)

That schema then goes directly into the generation config:

val model = Firebase.ai(backend = GenerativeBackend.googleAI())
    .generativeModel(
        modelName = "gemini-3.5-flash",
        tools = listOf(Tool.googleMaps()),
        generationConfig = generationConfig {
            responseMimeType = "application/json"
            responseSchema = responseJsonSchema
        }
    )

The response comes back as a JSON string you can deserialize directly into your data classes with kotlinx.serialization or Moshi. No prompt engineering. No fallback parsing. No "sometimes the model adds a preamble" bugs in production.

2. Google Maps Grounding — Solving the Hallucination Problem at the Source

If you've tried using an LLM to generate location-based content, you've seen hallucinated coordinates. Ask for nearby restaurants and you get addresses that don't exist, names that are almost right, a "4.8-star pizza place" that Google Maps has never heard of. For consumer apps this is embarrassing. For a navigation or tour guide app it's a trust killer.

Maps Grounding is how Firebase AI Logic solves this. You pass Tool.googleMaps() in the tools list and a RetrievalConfig with the user's actual latitude and longitude. The model then has access to the Google Maps index for that location, and any place names, addresses, or coordinates in its response are grounded to real entities in Maps data rather than generated from parametric memory.

val configForTools = ToolConfig(
    functionCallingConfig = null,
    retrievalConfig = retrievalConfig {
        latLng = FirebaseLatLng(pose.latitude, pose.longitude)
        languageCode = "en"
    }
)

val result = model.generateContent(
    content {
        text(
            "The user is at latitude ${pose.latitude} and longitude ${pose.longitude}. " +
            "Generate exactly 3 diverse tours near this location. " +
            "All tour stops should be walking distance only."
        )
    },
    requestOptions = RequestOptions(toolConfig = configForTools)
)

The grounding happens server-side — the model calls into Maps as a tool during generation, which is why you need to pass the coordinates in the retrieval config rather than just in the prompt text. Putting coordinates in the prompt alone doesn't give the model access to the Maps index; it just tells it where you are.

For apps like Samachar where I've thought about adding location-aware news or event recommendations, this would directly replace the category of bugs that come from Gemini generating content about places it's confabulated from training. The pattern is clean: structured output schema + Maps grounding = LLM-generated location content you can actually trust enough to show users.

3. Gemini 2.5 Flash TTS — Native Audio Generation

The third pattern in the tour guide app is one I hadn't seen documented clearly before: using Gemini 2.5 Flash as a text-to-speech model directly in the Firebase AI Logic SDK, with audio bytes returned inline rather than streamed to a URL.

The key is ResponseModality.AUDIO in the generation config:

val ttsModel = Firebase.ai(backend = GenerativeBackend.googleAI())
    .generativeModel(
        modelName = "gemini-2.5-flash-tts",
        generationConfig = generationConfig {
            responseModalities = listOf(ResponseModality.AUDIO)
        }
    )

val response = ttsModel.generateContent(
    "Say in a neutral but positive voice:\n$tourDescription"
)

val audioBytes = response.candidates.firstOrNull()?.content?.parts
    ?.filterIsInstance()
    ?.firstOrNull { it.mimeType.contains("audio") }?.inlineData

The audio comes back as raw bytes in an InlineDataPart, which you can pipe straight into an AudioTrack or write to a temp file and play through ExoPlayer or MediaPlayer. The MIME type in the response tells you the format — you'd want to check that before assuming PCM.

For a news app like Samachar, this opens up a "read this article to me" feature with a naturalness bar that on-device TTS can't match. For Musist, where I spent significant effort integrating AdMob creatives with audio, having AI-generated narration at this quality level via a single API call changes the production economics entirely.

The Architecture Under the Hood

One thing the tour guide app makes concrete is how to chain these patterns in a sensible data flow. The three stages don't run in parallel — each feeds the next:

1. Location first: the ARCore Geospatial API provides a GeospatialPose with sub-meter accuracy latitude, longitude, and heading. You wait for horizontalAccuracy to drop below your threshold before proceeding — running the rest of the pipeline on a coarse GPS fix defeats the purpose.
2. Content from Gemini: the pose coordinates go into the retrieval config for Maps Grounding, and the structured schema ensures you get back a typed object rather than text. No I/O, no database — just an API call that returns your domain model.
3. Audio from TTS: the generated tour descriptions go to Gemini 2.5 Flash TTS, which returns audio bytes. These get played through the device speaker or cached to disk for offline playback.

The XR rendering layer (SpatialBox, SceneCoreEntity, AnimatedSpatialVisibility, InteractableComponent) is on top of this and is XR-specific — but steps 1 through 3 are standard Android code that would work identically in a conventional phone app. The Geospatial API works in camera mode on regular Android as well as in full XR mode.

What This Means for Non-XR Apps

If you're an Android developer who's not building XR experiences — which is most of us, at least for now — these three patterns are still immediately useful:

• Structured output schemas replace fragile JSON extraction in any feature that calls Gemini for structured data: recommendations, summaries, entity extraction, classification. Define the schema once, parse the result once, done.
• Maps Grounding makes location-based AI features shippable for apps that already use location data. Restaurant finders, local event generators, nearby POI descriptions — the class of features where hallucination has historically been a dealbreaker.
• Gemini 2.5 Flash TTS is a plausible replacement for on-device TextToSpeech wherever audio quality matters more than latency: article narration, accessibility features for visually impaired users, AI-generated audio guides.

What to watch for in production

The latency profile of this stack is not what you'd get from a local model or cached content. Gemini 3.5 Flash with Maps Grounding is faster than the larger models, but it's still a round-trip to a remote API — you want to trigger generation ahead of need when you can predict the user's next action, not on-demand in a hot path. In the tour guide app, this is baked into the design: generation happens when the user arrives at a location, not when they tap a "play" button.

Maps Grounding currently requires an internet connection — there's no offline mode for the maps index. For apps with offline requirements, you'd need a cached fallback. And the TTS audio bytes you get back are not tiny: budget for appropriate caching or streaming strategy rather than re-generating on every playback.

Should You Adopt Firebase AI Logic Now?

If you're adding Gemini features to a new Android project, yes — start here rather than wiring up the REST API directly. The SDK handles auth, quota, AppCheck, and response parsing in a way that would take real time to replicate yourself, and the Schema API alone justifies the dependency for anything beyond simple completions.

If you have an existing direct Gemini integration, migration is mechanical: the API shapes are similar, you're mostly swapping the initialization and model instantiation code. The structured output and grounding features are the main reason to bother — if you don't need either, there's no urgency.

The tour guide app from Google is more than a demo. It's a concrete, buildable reference for how three production patterns — structured AI output, grounded location content, and AI-generated audio — compose together in a real Android codebase. That's not always true of Google's sample apps, and it's worth pulling apart even if you never build anything for XR hardware.