Google Vertex AI

Google Vertex AI provides enterprise-grade access to Google's generative AI models with enhanced security, scaling capabilities, and integration with Google Cloud services.

Sample configuration

Google Vertex AI supports BYOK only. It isn't available in managed mode. The following example starts an ephemeral session with Vertex AI as the LLM. In an ephemeral session, you pass the agent configuration and your credential in the request, and the Platform doesn't save them. Agora sends the credential to Vertex AI as an OAuth 2.0 access token, in an Authorization: Bearer header. Get a token for your Google Cloud project, for example with gcloud auth print-access-token. For details, see REST authentication.

Call create_session() on an Agent. The SDK starts an ephemeral session when you set a Platform session option, such as max_duration in seconds.

from agora_agent import Agent, DeepgramSTT, OpenAITTS, VertexAILLM

test_agent = (
    Agent(client=client)
    .with_stt(DeepgramSTT(model="nova-3", language="en-US"))
    .with_llm(
        VertexAILLM(
            api_key="<your_access_token>",
            vendor="vertexai",
            project_id="<your_project_id>",
            location="global",
            model="gemini-3.8-flash",
            url="https://aiplatform.googleapis.com/v1/projects/<your_project_id>/locations/global/publishers/google/models/gemini-3.8-flash:streamGenerateContent?alt=sse",
            max_history=32,
            failure_message="Hold on a second.",
            system_messages=[
                {"role": "system", "content": "You are a friendly support agent for Acme. Keep your answers short."}
            ],
        )
    )
    .with_tts(OpenAITTS(model="tts-1", voice="alloy"))
)

session = test_agent.create_session(
    channel="<your_channel_name>",
    token="<your_rtc_token>",
    agent_uid="1002",
    remote_uids=["1001"],
    max_duration=600,
)
session_id = session.start()

Call createSession() on an Agent. The SDK starts an ephemeral session when you set a Platform session option, such as maxDuration in seconds.

import { Agent, DeepgramSTT, OpenAITTS, VertexAILLM } from 'agora-agents';

const testAgent = new Agent({ client })
  .withStt(new DeepgramSTT({ model: 'nova-3', language: 'en-US' }))
  .withLlm(
    new VertexAILLM({
      apiKey: '<your_access_token>',
      vendor: 'vertexai',
      projectId: '<your_project_id>',
      location: 'global',
      model: 'gemini-3.8-flash',
      url: 'https://aiplatform.googleapis.com/v1/projects/<your_project_id>/locations/global/publishers/google/models/gemini-3.8-flash:streamGenerateContent?alt=sse',
      maxHistory: 32,
      failureMessage: 'Hold on a second.',
      systemMessages: [
        { role: 'system', content: 'You are a friendly support agent for Acme. Keep your answers short.' },
      ],
    }),
  )
  .withTts(new OpenAITTS({ model: 'tts-1', voice: 'alloy' }));

const session = testAgent.createSession({
  channel: '<your_channel_name>',
  token: '<your_rtc_token>',
  agentUid: '1002',
  remoteUids: ['1001'],
  maxDuration: 600,
});
const sessionId = await session.start();

Call CreateSession() on an Agent. The SDK starts an ephemeral session when you set a Platform session option, such as MaxDuration in seconds.

maxHistory := 32
testAgent := agentkit.NewAgent(client).
	WithStt(vendors.NewDeepgramSTT(vendors.DeepgramSTTOptions{Model: "nova-3", Language: "en-US"})).
	WithLlm(vendors.NewVertexAILLM(vendors.VertexAILLMOptions{
		GeminiOptions: vendors.GeminiOptions{
			APIKey:         "<your_access_token>",
			Vendor:         "vertexai",
			Model:          "gemini-3.8-flash",
			URL:            "https://aiplatform.googleapis.com/v1/projects/<your_project_id>/locations/global/publishers/google/models/gemini-3.8-flash:streamGenerateContent?alt=sse",
			MaxHistory:     &maxHistory,
			FailureMessage: "Hold on a second.",
			SystemMessages: []map[string]interface{}{
				{"role": "system", "content": "You are a friendly support agent for Acme. Keep your answers short."},
			},
		},
		ProjectID: "<your_project_id>",
		Location:  "global",
	})).
	WithTts(vendors.NewOpenAITTS(vendors.OpenAITTSOptions{Model: "tts-1", Voice: "alloy"}))

maxDuration := 600
session := testAgent.CreateSession(agentkit.CreateSessionOptions{
	Channel:     "<your_channel_name>",
	Token:       "<your_rtc_token>",
	AgentUID:    "1002",
	RemoteUIDs:  []string{"1001"},
	MaxDuration: &maxDuration,
})
sessionID, err := session.Start(ctx)
if err != nil {
	log.Fatal(err)
}

Send a request to Start an ephemeral session. In the llm block, set credential.mode to byok and credential.fields.api_key to the access token. Your project ID and location aren't secret. Pass them in the url, replacing <your_project_id>. The following example uses the global endpoint:

curl --request POST \
  --url https://preview.ai.agora.io/conversational-ai/v1/projects/<your_app_id>/sessions/ephemeral \
  --header 'Authorization: Basic <your_base64_encoded_credentials>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "agent": {
    "instructions": "You are a friendly support agent for Acme. Keep your answers short.",
    "pipeline": {
      "mode": "cascaded",
      "asr": { "credential": { "mode": "managed" }, "vendor": "deepgram", "model": "nova-3", "language": "en-US" },
      "llm": {
        "credential": { "mode": "byok", "fields": { "api_key": "<your_access_token>" } },
        "vendor": "vertexai",
        "style": "gemini",
        "model": "gemini-3.8-flash",
        "url": "https://aiplatform.googleapis.com/v1/projects/<your_project_id>/locations/global/publishers/google/models/gemini-3.8-flash:streamGenerateContent?alt=sse",
        "max_history": 32,
        "failure_message": "Hold on a second."
      },
      "tts": {
        "credential": { "mode": "managed" },
        "vendor": "minimax",
        "params": { "model": "speech-2.6-turbo", "voice_setting": { "voice_id": "English_captivating_female1", "speed": 1 } }
      }
    }
  },
  "transport": {
    "type": "rtc",
    "channel": "<your_channel_name>",
    "token": "<your_rtc_token>",
    "uid": "1002",
    "subscribe_uids": ["1001"]
  }
}'

The response has the same format as Start a session. Set the system prompt and the greeting with the agent's instructions and greeting, not in the llm block.

Key parameters

The llm block takes the following fields. In the SDKs, the LLM class sets them from its options. Use the VertexAILLM class. The SDK builds the endpoint URL from the project ID, the location, and the model.

llm

vendorstring

required

Set to vertexai.

credentialobject

required

Set mode to byok and fields.api_key to an OAuth 2.0 access token for your Google Cloud project. Don't put the token in the url.

stylestring

required

Set to gemini to use Gemini's message format.

modelstring

required

Refer to Vertex AI models for available models.

urlstring

required

The Vertex AI streamGenerateContent endpoint, with your Google Cloud project ID, location, and model in the path. Use one of these hosts, with the matching location in the path:

Global: aiplatform.googleapis.com with locations/global.
US multi-region: aiplatform.us.rep.googleapis.com with locations/us.
EU multi-region: aiplatform.eu.rep.googleapis.com with locations/eu.
A region: <region>-aiplatform.googleapis.com with locations/<region>, for example us-central1.

Some models aren't offered in every location. If a request returns 404, use the global endpoint. Include alt=sse in the URL. For details, see Vertex AI locations.

For complete llm block options, see Create an agent. For advanced configuration options, model capabilities, and detailed parameter descriptions, see the Google Vertex AI API documentation.

Sample configuration

Key parameters

Contact Us

Contact Us

Sample configurationCopy Anchor Link

Key parametersCopy Anchor Link

Contact Us

Contact Us

Sample configuration

Key parameters