Google Gemini
Updated
Use Google Gemini as the LLM in a Conversational AI Platform agent.
Preview documentation
Google Gemini provides advanced multimodal AI capabilities with fast performance and efficient processing for conversational AI applications.
Sample configuration
Google Gemini supports BYOK only. It isn't available in managed mode. The following example starts an ephemeral session with Gemini as the LLM. In an ephemeral session, you pass the agent configuration in the request, with your Gemini API key in the endpoint URL, and the Platform doesn't save them. Get your API key from Google AI Studio.
Pass your key as api_key, and leave it out of url. Call create_session() on an Agent. The SDK starts an ephemeral session when you set a Platform session option, such as max_duration in seconds.
from agora_agent import Agent, DeepgramSTT, Gemini, OpenAITTS
test_agent = (
Agent(client=client)
.with_stt(DeepgramSTT(model="nova-3", language="en-US"))
.with_llm(
Gemini(
api_key="<your_gemini_api_key>",
model="gemini-3.6-flash",
url="https://generativelanguage.googleapis.com/v1beta/models/gemini-3.6-flash:streamGenerateContent?alt=sse",
max_history=32,
failure_message="Hold on a second.",
system_messages=[
{"role": "system", "content": "You are a friendly support agent for Acme. Keep your answers short."}
],
)
)
.with_tts(OpenAITTS(model="tts-1", voice="alloy"))
)
session = test_agent.create_session(
channel="<your_channel_name>",
token="<your_rtc_token>",
agent_uid="1002",
remote_uids=["1001"],
max_duration=600,
)
session_id = session.start()Pass your key as apiKey, and leave it out of url. Call createSession() on an Agent. The SDK starts an ephemeral session when you set a Platform session option, such as maxDuration in seconds.
import { Agent, DeepgramSTT, Gemini, OpenAITTS } from 'agora-agents';
const testAgent = new Agent({ client })
.withStt(new DeepgramSTT({ model: 'nova-3', language: 'en-US' }))
.withLlm(
new Gemini({
apiKey: '<your_gemini_api_key>',
model: 'gemini-3.6-flash',
url: 'https://generativelanguage.googleapis.com/v1beta/models/gemini-3.6-flash:streamGenerateContent?alt=sse',
maxHistory: 32,
failureMessage: 'Hold on a second.',
systemMessages: [
{ role: 'system', content: 'You are a friendly support agent for Acme. Keep your answers short.' },
],
}),
)
.withTts(new OpenAITTS({ model: 'tts-1', voice: 'alloy' }));
const session = testAgent.createSession({
channel: '<your_channel_name>',
token: '<your_rtc_token>',
agentUid: '1002',
remoteUids: ['1001'],
maxDuration: 600,
});
const sessionId = await session.start();Pass your key as APIKey, and leave it out of URL. Call CreateSession() on an Agent. The SDK starts an ephemeral session when you set a Platform session option, such as MaxDuration in seconds.
maxHistory := 32
testAgent := agentkit.NewAgent(client).
WithStt(vendors.NewDeepgramSTT(vendors.DeepgramSTTOptions{Model: "nova-3", Language: "en-US"})).
WithLlm(vendors.NewGemini(vendors.GeminiOptions{
APIKey: "<your_gemini_api_key>",
Model: "gemini-3.6-flash",
URL: "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.6-flash:streamGenerateContent?alt=sse",
MaxHistory: &maxHistory,
FailureMessage: "Hold on a second.",
SystemMessages: []map[string]interface{}{
{"role": "system", "content": "You are a friendly support agent for Acme. Keep your answers short."},
},
})).
WithTts(vendors.NewOpenAITTS(vendors.OpenAITTSOptions{Model: "tts-1", Voice: "alloy"}))
maxDuration := 600
session := testAgent.CreateSession(agentkit.CreateSessionOptions{
Channel: "<your_channel_name>",
Token: "<your_rtc_token>",
AgentUID: "1002",
RemoteUIDs: []string{"1001"},
MaxDuration: &maxDuration,
})
sessionID, err := session.Start(ctx)
if err != nil {
log.Fatal(err)
}Send a request to Start an ephemeral session. In the llm block, set url to the Gemini streamGenerateContent endpoint for your model, with alt=sse and your API key in the key query parameter. Set credential.mode to byok, and leave out credential.fields.
curl --request POST \
--url https://preview.ai.agora.io/conversational-ai/v1/projects/<your_app_id>/sessions/ephemeral \
--header 'Authorization: Basic <your_base64_encoded_credentials>' \
--header 'Content-Type: application/json' \
--data '
{
"agent": {
"instructions": "You are a friendly support agent for Acme. Keep your answers short.",
"pipeline": {
"mode": "cascaded",
"asr": { "credential": { "mode": "managed" }, "vendor": "deepgram", "model": "nova-3", "language": "en-US" },
"llm": {
"credential": { "mode": "byok" },
"vendor": "gemini",
"style": "gemini",
"model": "gemini-3.6-flash",
"url": "https://generativelanguage.googleapis.com/v1beta/models/gemini-3.6-flash:streamGenerateContent?alt=sse&key=<your_gemini_api_key>",
"max_history": 32,
"failure_message": "Hold on a second."
},
"tts": {
"credential": { "mode": "managed" },
"vendor": "minimax",
"params": { "model": "speech-2.6-turbo", "voice_setting": { "voice_id": "English_captivating_female1", "speed": 1 } }
}
}
},
"transport": {
"type": "rtc",
"channel": "<your_channel_name>",
"token": "<your_rtc_token>",
"uid": "1002",
"subscribe_uids": ["1001"]
}
}'A saved agent can't have a key in its url. The response has the same format as Start a session. Set the system prompt and the greeting with the agent's instructions and greeting, not in the llm block.
Key parameters
The llm block takes the following fields.
vendorstringSet to gemini.
credentialobjectSet mode to byok.
stylestringSet to gemini to use Gemini's message format.
modelstringRefer to Gemini models for available models.
urlstringThe Gemini streamGenerateContent endpoint for your model, with alt=sse and key=<your_gemini_api_key>. Get your API key from Google AI Studio.
paramsobjectignore_emptybooleanSet to true to handle empty responses appropriately.
For complete llm block options, see Create an agent. For advanced configuration options, model capabilities, and detailed parameter descriptions, see the Google Gemini API documentation.