Google Gemini Live
Updated
Integrate Google Gemini Live with the Conversational AI Engine using the Gemini Developer API.
Agora has partnered with Google Gemini Live to bring frontier multimodal large language model capabilities to your app. Gemini Live provides real-time audio processing, enabling natural voice conversations without separate ASR and TTS components. This page explains how to integrate Gemini Live through the Gemini Developer API. Enabling MLLM automatically disables ASR, LLM, and TTS since the MLLM handles end-to-end voice processing directly.
Supported models
The following Gemini Live models are supported:
| Model | Description |
|---|---|
models/gemini-3.8-live | General-purpose Gemini Live model. |
models/gemini-3.8-live-extended-thinking | Gemini Live model with an adjustable reasoning budget. |
Use models/gemini-3.8-live-extended-thinking when the conversation benefits from additional reasoning. Set thinking_level to low, medium, or high. Omit thinking_level when using models/gemini-3.8-live.
Sample configuration
The following examples show how to configure Google Gemini Live MLLM (Gemini Developer API) when starting a conversational AI agent.
from agora_agent import Agent
from agora_agent.agentkit.vendors import GeminiLive
# client is your configured Agora client
agent = (
Agent(client)
.with_mllm(GeminiLive(
api_key='your-google-api-key',
model='models/gemini-3.8-live-extended-thinking',
thinking_level='medium',
voice='Charon',
instructions='You are a friendly assistant.',
transcribe_agent=True,
transcribe_user=True,
))
)import { Agent, GeminiLive } from 'agora-agents';
// client is your configured Agora client
const agent = new Agent({ client })
.withMllm(new GeminiLive({
apiKey: 'your-google-api-key',
model: 'models/gemini-3.8-live-extended-thinking',
thinkingLevel: 'medium',
voice: 'Charon',
instructions: 'You are a friendly assistant.',
transcribeAgent: true,
transcribeUser: true,
}));import (
Agora "github.com/AgoraIO/agora-agents-go/v2"
"github.com/AgoraIO/agora-agents-go/v2/agentkit"
"github.com/AgoraIO/agora-agents-go/v2/agentkit/vendors"
)
// client is your configured Agora client
agent := agentkit.NewAgent(client).WithMllm(
vendors.NewGeminiLive(vendors.GeminiLiveOptions{
APIKey: "your-google-api-key",
Model: "models/gemini-3.8-live-extended-thinking",
ThinkingLevel: vendors.GeminiThinkingLevelMedium,
Voice: "Charon",
Instructions: "You are a friendly assistant.",
TranscribeAgent: Agora.Bool(true),
TranscribeUser: Agora.Bool(true),
}),
)Use the following mllm configuration in your request:
"mllm": {
"enable": true,
"api_key": "<GOOGLE_GEMINI_API_KEY>",
"messages": [
{
"role": "user",
"content": "<HISTORY_CONTENT>"
}
],
"params": {
"model": "models/gemini-3.8-live-extended-thinking",
"thinking_level": "medium",
"instructions": "You are a friendly assistant.",
"voice": "Charon",
"affective_dialog": false,
"proactive_audio": false,
"transcribe_agent": true,
"transcribe_user": true,
"http_options": {
"api_version": "v1beta"
}
},
"turn_detection": {
// see details below
},
"input_modalities": [
"audio"
],
"output_modalities": [
"audio"
],
"greeting_message": "Hi, how can I assist you today?",
"failure_message": "Sorry, I encountered an issue. Please try again.",
"vendor": "gemini"
}Turn detection
To set up turn detection, add a turn_detection block inside the mllm object when you Start a conversational AI agent.
Info
When mllm.turn_detection is defined, the top-level turn_detection object has no effect.
The following examples show the supported configurations for Google Gemini Live.
-
Server VAD
"turn_detection": { "mode": "server_vad", "server_vad_config": { "prefix_padding_ms": 800, "silence_duration_ms": 640, "start_of_speech_sensitivity": "START_SENSITIVITY_HIGH", "end_of_speech_sensitivity": "END_SENSITIVITY_HIGH" } } -
Agora VAD
"turn_detection": { "mode": "agora_vad", "agora_vad_config": { "interrupt_duration_ms": 160, "prefix_padding_ms": 800, "silence_duration_ms": 640, "threshold": 0.5 } }
Key parameters
| Parameter | Description |
|---|---|
enableboolean | Enables the MLLM module. Replaces the deprecated |
api_keystring | The Google Gemini API key used to authenticate requests. You can generate an API key in Google AI Studio. |
messagesarray[object] | An array of conversation history items passed to the model as context. Each item represents a single message in the conversation history.
|
paramsobject | Configuration object for the Gemini Live model.
|
turn_detectionobject | Turn detection configuration for the MLLM module. For a full list of
|
input_modalitiesarray[string] |
Input modalities for the MLLM.
|
output_modalitiesarray[string] |
Output modalities for the MLLM.
|
greeting_messagestring | The message the agent speaks when a user joins the channel. |
failure_messagestring | The message the agent speaks when an error occurs. |
vendorstring | The MLLM provider identifier. Set to |
For comprehensive API reference, real-time capabilities, and detailed parameter descriptions, see the Google Gemini Live API.
