# Google Vertex AI (/en/ai/platform/models/llm/google-vertex-ai)

> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).

> Documentation: Conversational AI.
> Documentation version: Preview.
> This document describes only this version. [Version index](/llms/doc-versions-convoai-guides-v2.txt).


Google Vertex AI provides enterprise-grade access to Google's generative AI models with enhanced security, scaling capabilities, and integration with Google Cloud services.

## Sample configuration

Google Vertex AI supports BYOK only. It isn't available in managed mode. The following example starts an [ephemeral session](/en/ai/platform/get-started/start-stop-agent#test-a-configuration-without-saving-an-agent) with Vertex AI as the LLM. In an ephemeral session, you pass the agent configuration and your credential in the request, and the Platform doesn't save them. Agora sends the credential to Vertex AI as an OAuth 2.0 access token, in an `Authorization: Bearer` header. Get a token for your Google Cloud project, for example with `gcloud auth print-access-token`. For details, see [REST authentication](https://cloud.google.com/docs/authentication/rest).

**Python SDK**

Call `create_session()` on an `Agent`. The SDK starts an ephemeral session when you set a Platform session option, such as `max_duration` in seconds.

```python
from agora_agent import Agent, DeepgramSTT, OpenAITTS, VertexAILLM

test_agent = (
    Agent(client=client)
    .with_stt(DeepgramSTT(model="nova-3", language="en-US"))
    .with_llm(
        VertexAILLM(
            api_key="<your_access_token>",
            vendor="vertexai",
            project_id="<your_project_id>",
            location="global",
            model="gemini-3.8-flash",
            url="https://aiplatform.googleapis.com/v1/projects/<your_project_id>/locations/global/publishers/google/models/gemini-3.8-flash:streamGenerateContent?alt=sse",
            max_history=32,
            failure_message="Hold on a second.",
            system_messages=[
                {"role": "system", "content": "You are a friendly support agent for Acme. Keep your answers short."}
            ],
        )
    )
    .with_tts(OpenAITTS(model="tts-1", voice="alloy"))
)

session = test_agent.create_session(
    channel="<your_channel_name>",
    token="<your_rtc_token>",
    agent_uid="1002",
    remote_uids=["1001"],
    max_duration=600,
)
session_id = session.start()
```

**TypeScript SDK**

Call `createSession()` on an `Agent`. The SDK starts an ephemeral session when you set a Platform session option, such as `maxDuration` in seconds.

```ts
import { Agent, DeepgramSTT, OpenAITTS, VertexAILLM } from 'agora-agents';

const testAgent = new Agent({ client })
  .withStt(new DeepgramSTT({ model: 'nova-3', language: 'en-US' }))
  .withLlm(
    new VertexAILLM({
      apiKey: '<your_access_token>',
      vendor: 'vertexai',
      projectId: '<your_project_id>',
      location: 'global',
      model: 'gemini-3.8-flash',
      url: 'https://aiplatform.googleapis.com/v1/projects/<your_project_id>/locations/global/publishers/google/models/gemini-3.8-flash:streamGenerateContent?alt=sse',
      maxHistory: 32,
      failureMessage: 'Hold on a second.',
      systemMessages: [
        { role: 'system', content: 'You are a friendly support agent for Acme. Keep your answers short.' },
      ],
    }),
  )
  .withTts(new OpenAITTS({ model: 'tts-1', voice: 'alloy' }));

const session = testAgent.createSession({
  channel: '<your_channel_name>',
  token: '<your_rtc_token>',
  agentUid: '1002',
  remoteUids: ['1001'],
  maxDuration: 600,
});
const sessionId = await session.start();
```

**Go SDK**

Call `CreateSession()` on an `Agent`. The SDK starts an ephemeral session when you set a Platform session option, such as `MaxDuration` in seconds.

```go
maxHistory := 32
testAgent := agentkit.NewAgent(client).
	WithStt(vendors.NewDeepgramSTT(vendors.DeepgramSTTOptions{Model: "nova-3", Language: "en-US"})).
	WithLlm(vendors.NewVertexAILLM(vendors.VertexAILLMOptions{
		GeminiOptions: vendors.GeminiOptions{
			APIKey:         "<your_access_token>",
			Vendor:         "vertexai",
			Model:          "gemini-3.8-flash",
			URL:            "https://aiplatform.googleapis.com/v1/projects/<your_project_id>/locations/global/publishers/google/models/gemini-3.8-flash:streamGenerateContent?alt=sse",
			MaxHistory:     &maxHistory,
			FailureMessage: "Hold on a second.",
			SystemMessages: []map[string]interface{}{
				{"role": "system", "content": "You are a friendly support agent for Acme. Keep your answers short."},
			},
		},
		ProjectID: "<your_project_id>",
		Location:  "global",
	})).
	WithTts(vendors.NewOpenAITTS(vendors.OpenAITTSOptions{Model: "tts-1", Voice: "alloy"}))

maxDuration := 600
session := testAgent.CreateSession(agentkit.CreateSessionOptions{
	Channel:     "<your_channel_name>",
	Token:       "<your_rtc_token>",
	AgentUID:    "1002",
	RemoteUIDs:  []string{"1001"},
	MaxDuration: &maxDuration,
})
sessionID, err := session.Start(ctx)
if err != nil {
	log.Fatal(err)
}
```

**REST API**

Send a request to [Start an ephemeral session](/en/api-reference/api-ref/conversational-ai/platform/start-ephemeral-session). In the `llm` block, set `credential.mode` to `byok` and `credential.fields.api_key` to the access token. Your project ID and location aren't secret. Pass them in the `url`, replacing `<your_project_id>`. The following example uses the global endpoint:

```bash
curl --request POST \
  --url https://preview.ai.agora.io/conversational-ai/v1/projects/<your_app_id>/sessions/ephemeral \
  --header 'Authorization: Basic <your_base64_encoded_credentials>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "agent": {
    "instructions": "You are a friendly support agent for Acme. Keep your answers short.",
    "pipeline": {
      "mode": "cascaded",
      "asr": { "credential": { "mode": "managed" }, "vendor": "deepgram", "model": "nova-3", "language": "en-US" },
      "llm": {
        "credential": { "mode": "byok", "fields": { "api_key": "<your_access_token>" } },
        "vendor": "vertexai",
        "style": "gemini",
        "model": "gemini-3.8-flash",
        "url": "https://aiplatform.googleapis.com/v1/projects/<your_project_id>/locations/global/publishers/google/models/gemini-3.8-flash:streamGenerateContent?alt=sse",
        "max_history": 32,
        "failure_message": "Hold on a second."
      },
      "tts": {
        "credential": { "mode": "managed" },
        "vendor": "minimax",
        "params": { "model": "speech-2.6-turbo", "voice_setting": { "voice_id": "English_captivating_female1", "speed": 1 } }
      }
    }
  },
  "transport": {
    "type": "rtc",
    "channel": "<your_channel_name>",
    "token": "<your_rtc_token>",
    "uid": "1002",
    "subscribe_uids": ["1001"]
  }
}'
```

The response has the same format as Start a session. Set the system prompt and the greeting with the agent's `instructions` and `greeting`, not in the `llm` block.

## Key parameters

The `llm` block takes the following fields. In the SDKs, the LLM class sets them from its options. Use the `VertexAILLM` class. The SDK builds the endpoint URL from the project ID, the location, and the model.

**llm**

**`vendor` (string) required**

Set to `vertexai`.

**`credential` (object) required**

Set `mode` to `byok` and `fields.api_key` to an OAuth 2.0 access token for your Google Cloud project. Don't put the token in the `url`.

**`style` (string) required**

Set to `gemini` to use Gemini's message format.

**`model` (string) required**

Refer to [Vertex AI models](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/models) for available models.

**`url` (string) required**

The Vertex AI `streamGenerateContent` endpoint, with your Google Cloud project ID, location, and model in the path. Use one of these hosts, with the matching location in the path:

- Global: `aiplatform.googleapis.com` with `locations/global`.
- US multi-region: `aiplatform.us.rep.googleapis.com` with `locations/us`.
- EU multi-region: `aiplatform.eu.rep.googleapis.com` with `locations/eu`.
- A region: `<region>-aiplatform.googleapis.com` with `locations/<region>`, for example `us-central1`.

Some models aren't offered in every location. If a request returns `404`, use the global endpoint. Include `alt=sse` in the URL. For details, see [Vertex AI locations](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/locations).

For complete `llm` block options, see [Create an agent](/en/api-reference/api-ref/conversational-ai/platform/create-agent). For advanced configuration options, model capabilities, and detailed parameter descriptions, see the [Google Vertex AI API documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/system-instructions).