# OpenAI GPT-Live (/en/ai/models/mllm/openai-gpt-live)

> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).

OpenAI GPT-Live provides multimodal large language model capabilities with real-time audio processing, enabling natural voice conversations without separate ASR/TTS components. Enabling MLLM automatically disables ASR, LLM, and TTS since the MLLM handles end-to-end voice processing directly.

### Sample configuration

The following examples show how to configure OpenAI GPT-Live MLLM when starting a conversational AI agent.

<Tabs defaultValue="python" groupId="ai-sdk-language">
  <TabsList>
    <TabsTrigger value="python">
      Python SDK
    </TabsTrigger>

    <TabsTrigger value="typescript">
      TypeScript SDK
    </TabsTrigger>

    <TabsTrigger value="go">
      Go SDK
    </TabsTrigger>

    <TabsTrigger value="rest-api">
      REST API
    </TabsTrigger>
  </TabsList>

  <TabsContent value="python">
    ```python
    from agora_agent import Agent, OpenAIGPTLive

    # client is your configured Agora client
    agent = (
        Agent(client)
        .with_mllm(OpenAIGPTLive(
            api_key='your-openai-key',
            greeting="Hello! I'm GPT-live. How can I help you today?",
            model='gpt-live-1',
            voice='marin',
            prompt='your-system-prompt',
        ))
    )
    ```
  </TabsContent>

  <TabsContent value="typescript">
    ```typescript
    import { Agent, OpenAIGPTLive } from 'agora-agents';

    // client is your configured Agora client
    const agent = new Agent({ client })
      .withMllm(new OpenAIGPTLive({
        apiKey: 'your-openai-key',
        greeting: "Hello! I'm GPT-live. How can I help you today?",
        model: 'gpt-live-1',
        voice: 'marin',
        prompt: 'your-system-prompt',
      }));
    ```
  </TabsContent>

  <TabsContent value="go">
    ```go
    import (
        "github.com/AgoraIO/agora-agents-go/v2/agentkit"
        "github.com/AgoraIO/agora-agents-go/v2/agentkit/vendors"
    )

    // client is your configured Agora client
    agent := agentkit.NewAgent(client).WithMllm(
        vendors.NewOpenAIGPTLive(vendors.OpenAIGPTLiveOptions{
            APIKey:          "your-openai-key",
            GreetingMessage: "Hello! I'm GPT-live. How can I help you today?",
            Model:           "gpt-live-1",
            Voice:           "marin",
            Prompt:          "your-system-prompt",
        }),
    )
    ```
  </TabsContent>

  <TabsContent value="rest-api">
    Use the following `mllm` configuration in your request:

    ```json
    "mllm": {
      "enable": true,
      "vendor": "openai_gpt_live",
      "api_key": "<openai_api_key>",
      "url": "wss://api.openai.com/v1/live/sessions",
      "greeting_message": "Hello! I'm GPT-live. How can I help you today?",
      "params": {
        "model": "gpt-live-1",
        "voice": "marin",
        "prompt": "<system_prompt>"
      }
    }
    ```
  </TabsContent>
</Tabs>

### Tool calling

GPT-Live can call tools defined directly on the MLLM configuration. Set `mllm.mcp_servers`, enable `advanced_features.enable_tools`, and set `params.tool_enabled` to `true` to advertise those tools to GPT-Live:

```json
"mllm": {
  "mcp_servers": [
    {
      "name": "lookup",
      "endpoint": "https://tools.example/mcp",
      "transport": "streamable_http"
    }
  ],
  "params": {
    "tool_enabled": true
  }
},
"advanced_features": {
  "enable_tools": true
}
```

### Key parameters

<ParameterList title="mllm" required="true">
  <Parameter name="enable" type="boolean" required="false">
    Enables the MLLM module. Replaces the deprecated `advanced_features.enable_mllm`.
  </Parameter>

  <Parameter name="vendor" type="string" required="true">
    MLLM provider identifier. Set to `openai_gpt_live` for OpenAI GPT-Live.
  </Parameter>

  <Parameter name="api_key" type="string" required="true">
    The OpenAI API key used for authentication.
  </Parameter>

  <Parameter name="url" type="string" required="false" defaultValue="wss://api.openai.com/v1/live/sessions">
    The WebSocket URL for OpenAI GPT-Live.
  </Parameter>

  <Parameter name="messages" type="array[object]" required="false">
    An array of conversation history items passed to the model as context. Each item represents a single message in the conversation history.

    <Parameter name="role" type="string" required="true">
      The role of the message author. For example, `system` or `user`.
    </Parameter>

    <Parameter name="content" type="string" required="true">
      The content of the message.
    </Parameter>
  </Parameter>

  <Parameter name="input_modalities" type="array[string]" defaultValue="[&#x22;audio&#x22;]" required="false">
    MLLM input modalities:

    * `["audio"]`: Audio only
    * `["audio", "text"]`: Audio plus text
  </Parameter>

  <Parameter name="output_modalities" type="array[string]" defaultValue="[&#x22;text&#x22;, &#x22;audio&#x22;]" required="false">
    Output format options: `["text", "audio"]` for both text and voice responses.
  </Parameter>

  <Parameter name="greeting_message" type="string" required="false">
    Initial message the agent speaks when a user joins the channel.
  </Parameter>

  <Parameter name="failure_message" type="string" required="false">
    The message the agent speaks when an error occurs.
  </Parameter>

  <Parameter name="mcp_servers" type="array[object]" required="false">
    MCP servers whose tools GPT-Live can call. See [Tool calling](#tool-calling).
  </Parameter>

  <Parameter name="params" type="object" required="false">
    Additional MLLM configuration parameters.

    <Parameter name="model" type="string" required="false" defaultValue="gpt-live-1">
      The model identifier. Specify another supported model to override the default.
    </Parameter>

    <Parameter name="voice" type="string" required="false" defaultValue="marin">
      The voice identifier for audio output.
    </Parameter>

    <Parameter name="prompt" type="string" required="false">
      Session instructions that define the assistant's behavior and personality.
    </Parameter>

    <Parameter name="tool_enabled" type="boolean" required="false" defaultValue="false">
      Advertises the agent's graph tools to GPT-Live for tool calling. See [Tool calling](#tool-calling). Does not control delegate built-in tools.
    </Parameter>
  </Parameter>
</ParameterList>

<CalloutContainer type="warning">
  <CalloutTitle>
    Limitations
  </CalloutTitle>

  <CalloutDescription>
    GPT-Live doesn't support turn detection or input audio transcription.

    The parameters listed on this page are validated for use with Conversational AI Engine. Required parameters must be provided as documented. Additional parameters are passed through directly to the underlying vendor without validation.
  </CalloutDescription>
</CalloutContainer>