# Google Gemini Live (/en/ai/models/mllm/gemini)

> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).

Agora has [partnered with Google Gemini Live](https://ai.google.dev/gemini-api/docs/live-api#partner-integrations) to bring frontier multimodal large language model capabilities to your app. Gemini Live provides real-time audio processing, enabling natural voice conversations without separate ASR and TTS components. This page explains how to integrate Gemini Live through the Gemini Developer API.

<CalloutContainer type="info">
  <CalloutTitle>
    Info
  </CalloutTitle>

  <CalloutDescription>
    Enabling MLLM automatically disables ASR, LLM, and TTS since the MLLM handles end-to-end voice processing directly.
  </CalloutDescription>
</CalloutContainer>

### Sample configuration [#sample-configuration]

The following examples show how to configure Google Gemini Live MLLM (Gemini Developer API) when starting a conversational AI agent.

<Tabs defaultValue="python" groupId="ai-sdk-language">
  <TabsList>
    <TabsTrigger value="python">
      Python SDK
    </TabsTrigger>

    <TabsTrigger value="typescript">
      TypeScript SDK
    </TabsTrigger>

    <TabsTrigger value="go">
      Go SDK
    </TabsTrigger>

    <TabsTrigger value="rest-api">
      REST API
    </TabsTrigger>
  </TabsList>

  <TabsContent value="python">
    ```python
    from agora_agent import Agent
    from agora_agent.agentkit.vendors import GeminiLive

    # client is your configured Agora client
    agent = (
        Agent(client)
        .with_mllm(GeminiLive(
            api_key='your-google-api-key',
            model='gemini-3.1-flash-live-preview',
            voice='Charon',
            instructions='You are a friendly assistant.',
            transcribe_agent=True,
            transcribe_user=True,
        ))
    )
    ```
  </TabsContent>

  <TabsContent value="typescript">
    ```typescript
    import { Agent, GeminiLive } from 'agora-agents';

    // client is your configured Agora client
    const agent = new Agent({ client })
      .withMllm(new GeminiLive({
        apiKey: 'your-google-api-key',
        model: 'gemini-3.1-flash-live-preview',
        voice: 'Charon',
        instructions: 'You are a friendly assistant.',
        transcribeAgent: true,
        transcribeUser: true,
      }));
    ```
  </TabsContent>

  <TabsContent value="go">
    ```go
    import "github.com/AgoraIO/agora-agents-go/v2/agentkit/vendors"

    // client is your configured Agora client
    agent := agentkit.NewAgent(client).WithMllm(
        vendors.NewGeminiLive(vendors.GeminiLiveOptions{
            APIKey:         "your-google-api-key",
            Model:          "gemini-3.1-flash-live-preview",
            Voice:          "Charon",
            Instructions:   "You are a friendly assistant.",
            TranscribeAgent: agora.Bool(true),
            TranscribeUser:  agora.Bool(true),
        }),
    )
    ```
  </TabsContent>

  <TabsContent value="rest-api">
    Use the following `mllm` configuration in your request:

    ```json
    "mllm": {
      "enable": true,
      "api_key": "<GOOGLE_GEMINI_API_KEY>",
      "messages": [
        {
          "role": "user",
          "content": "<HISTORY_CONTENT>"
        }
      ],
      "params": {
        "model": "gemini-3.1-flash-live-preview",
        "instructions": "You are a friendly assistant.",
        "voice": "Charon",
        "affective_dialog": false,
        "proactive_audio": false,
        "transcribe_agent": true,
        "transcribe_user": true,
        "http_options": {
          "api_version": "v1beta"
        }
      },
      "turn_detection": {
        // see details below
      },
      "input_modalities": [
        "audio"
      ],
      "output_modalities": [
        "audio"
      ],
      "greeting_message": "Hi, how can I assist you today?",
      "failure_message": "Sorry, I encountered an issue. Please try again.",
      "vendor": "gemini"
    }
    ```
  </TabsContent>
</Tabs>

### Turn detection [#turn-detection]

To set up turn detection, add a `turn_detection` block inside the `mllm` object when you [Start a conversational AI agent](/en/api-reference/api-ref/conversational-ai/join).

<CalloutContainer type="info">
  <CalloutTitle>
    Info
  </CalloutTitle>

  <CalloutDescription>
    When `mllm.turn_detection` is defined, the top-level `turn_detection` object has no effect.
  </CalloutDescription>
</CalloutContainer>

The following examples show the supported configurations for Google Gemini Live.

* **Server VAD**

  ```json
  "turn_detection": {
    "mode": "server_vad",
    "server_vad_config": {
      "prefix_padding_ms": 800,
      "silence_duration_ms": 640,
      "start_of_speech_sensitivity": "START_SENSITIVITY_HIGH",
      "end_of_speech_sensitivity": "END_SENSITIVITY_HIGH"
    }
  }
  ```

* **Agora VAD**

  ```json
  "turn_detection": {
    "mode": "agora_vad",
    "agora_vad_config": {
      "interrupt_duration_ms": 160,
      "prefix_padding_ms": 800,
      "silence_duration_ms": 640,
      "threshold": 0.5
    }
  }
  ```

### Key parameters [#key-parameters]

<ParameterList title="mllm" required="true">
  <Parameter name="enable" type="boolean" required="false">
    Enables the MLLM module. Replaces the deprecated `advanced_features.enable_mllm`.
  </Parameter>

  <Parameter name="api_key" type="string" required="true">
    The Google Gemini API key used to authenticate requests. You can generate an API key in [Google AI Studio](https://aistudio.google.com/apikey).
  </Parameter>

  <Parameter name="messages" type="array[object]" required="false">
    An array of conversation history items passed to the model as context. Each item represents a single message in the conversation history.

    <Parameter name="role" type="string" required="true">
      The role of the message author. For example, `system` or `user`.
    </Parameter>

    <Parameter name="content" type="string" required="true">
      The content of the message.
    </Parameter>
  </Parameter>

  <Parameter name="params" type="object" required="true">
    Configuration object for the Gemini Live model.

    <Parameter name="model" type="string" required="true">
      The Gemini Live model identifier.
    </Parameter>

    <Parameter name="instructions" type="string" required="false">
      System instructions that define the agent's behavior or tone.
    </Parameter>

    <Parameter name="voice" type="string" required="false">
      The voice identifier for audio output. For example, `Aoede`, `Puck`, `Charon`, `Kore`, `Fenrir`, `Leda`, `Orus`, or `Zephyr`.
    </Parameter>

    <Parameter name="affective_dialog" type="boolean" required="false">
      Whether to enable affective dialog, which allows the model to adapt its tone based on the user's emotional cues.
    </Parameter>

    <Parameter name="proactive_audio" type="boolean" required="false">
      When enabled, the model may choose not to respond if the user's input does not require a reply, such as background speech or incomplete requests.
    </Parameter>

    <Parameter name="transcribe_agent" type="boolean" required="false">
      Whether to transcribe the agent's speech in real time.
    </Parameter>

    <Parameter name="transcribe_user" type="boolean" required="false">
      Whether to transcribe the user's speech in real time.
    </Parameter>

    <Parameter name="http_options" type="object" required="false">
      HTTP request options for the Gemini Live API.

      <Parameter name="api_version" type="string" required="false">
        The API version to use. For example, `v1beta`.
      </Parameter>
    </Parameter>
  </Parameter>

  <Parameter name="turn_detection" type="object" required="false">
    Turn detection configuration for the MLLM module. For a full list of `turn_detection` parameters, see [`mllm.turn_detection`](/en/api-reference/api-ref/conversational-ai/join#properties-mllm-turn-detection).

    <Parameter name="mode" type="string" required="false" possibleValues="agora_vad, server_vad">
      * `agora_vad`: Agora VAD-based detection.
      * `server_vad`: Vendor-side VAD-based detection.
    </Parameter>

    <Parameter name="agora_vad_config" type="object" required="false">
      Configuration for Agora VAD-based turn detection. Applicable when `mode` is `agora_vad`.

      <Parameter name="interrupt_duration_ms" type="integer" required="false">
        Minimum duration of speech in milliseconds required to trigger an interruption.
      </Parameter>

      <Parameter name="prefix_padding_ms" type="integer" required="false">
        Duration of audio in milliseconds to include before the detected speech start.
      </Parameter>

      <Parameter name="silence_duration_ms" type="integer" required="false">
        Duration of silence in milliseconds required to determine end of speech.
      </Parameter>

      <Parameter name="threshold" type="number" required="false">
        VAD sensitivity threshold. A higher value reduces false positives.
      </Parameter>
    </Parameter>

    <Parameter name="server_vad_config" type="object" required="false">
      Configuration for vendor-side VAD-based turn detection. Applicable when `mode` is `server_vad`. Parameters are passed through to the vendor.

      <Parameter name="prefix_padding_ms" type="integer" required="false">
        Duration of audio in milliseconds to include before the detected speech start.
      </Parameter>

      <Parameter name="silence_duration_ms" type="integer" required="false">
        Duration of silence in milliseconds required to determine end of speech.
      </Parameter>

      <Parameter name="start_of_speech_sensitivity" type="string" required="false" possibleValues="START_SENSITIVITY_HIGH,START_SENSITIVITY_LOW">
        Sensitivity for start of speech detection.
      </Parameter>

      <Parameter name="end_of_speech_sensitivity" type="string" required="false" possibleValues="END_SENSITIVITY_HIGH,END_SENSITIVITY_LOW">
        Sensitivity for end of speech detection.
      </Parameter>
    </Parameter>
  </Parameter>

  <Parameter name="input_modalities" type="array[string]" defaultValue="[&#x22;audio&#x22;]" required="false">
    Input modalities for the MLLM.

    * `["audio"]`: Audio-only input
    * `["audio", "text"]`: Accept both audio and text input
  </Parameter>

  <Parameter name="output_modalities" type="array[string]" defaultValue="[&#x22;audio&#x22;]" required="false">
    Output modalities for the MLLM.

    * `["audio"]`: Audio-only output
    * `["text", "audio"]`: Combined text and audio output
  </Parameter>

  <Parameter name="greeting_message" type="string" required="false">
    The message the agent speaks when a user joins the channel.
  </Parameter>

  <Parameter name="failure_message" type="string" required="false">
    The message the agent speaks when an error occurs.
  </Parameter>

  <Parameter name="vendor" type="string" required="true">
    The MLLM provider identifier. Set to `"gemini"` to use Google Gemini Live with the Gemini Developer API.
  </Parameter>
</ParameterList>

For comprehensive API reference, real-time capabilities, and detailed parameter descriptions, see the [Google Gemini Live API](https://ai.google.dev/api/live).