Groq
Updated
Use Groq as the LLM in a Conversational AI Platform agent.
Preview documentation
Groq provides state-of-the-art language models optimized for different use cases.
Sample configuration
The following example shows how to configure Groq as the agent's LLM when you create or update an agent. For a complete example, see Create an agent and start a session.
Groq supports BYOK only. It isn't available in managed mode. Store your Groq API key in a secret set, and refer to it from the agent. For details, see Use your own provider keys.
from agora_agent import Agent, Groq, secret_ref
# client is your configured Agora client
agent = (
Agent(client=client)
.with_stt(...) # configure your ASR vendor
.with_llm(
Groq(
api_key=secret_ref("support_prod", "groq"),
vendor="groq",
base_url="https://api.groq.com/openai/v1/chat/completions",
model="llama-3.3-70b-versatile",
max_history=32,
failure_message="Please hold on a second.",
system_messages=[
{"role": "system", "content": "You are a friendly support agent for Acme. Keep your answers short."}
],
)
)
.with_tts(...) # configure your TTS vendor
)import { Agent, Groq, secretRef } from 'agora-agents';
// client is your configured Agora client
const agent = new Agent({ client })
.withStt(/* configure your ASR vendor */)
.withLlm(
new Groq({
apiKey: secretRef('support_prod', 'groq'),
vendor: 'groq',
url: 'https://api.groq.com/openai/v1/chat/completions',
model: 'llama-3.3-70b-versatile',
maxHistory: 32,
failureMessage: 'Please hold on a second.',
systemMessages: [
{ role: 'system', content: 'You are a friendly support agent for Acme. Keep your answers short.' },
],
}),
)
.withTts(/* configure your TTS vendor */);// client is your configured Agora client
maxHistory := 32
agent := agentkit.NewAgent(client).
WithStt( /* configure your ASR vendor */ ).
WithLlm(vendors.NewGroq(vendors.GroqOptions{
APIKey: agentkit.SecretRef("support_prod", "groq"),
Vendor: "groq",
BaseURL: "https://api.groq.com/openai/v1/chat/completions",
Model: "llama-3.3-70b-versatile",
MaxHistory: &maxHistory,
FailureMessage: "Please hold on a second.",
SystemMessages: []map[string]interface{}{
{"role": "system", "content": "You are a friendly support agent for Acme. Keep your answers short."},
},
})).
WithTts( /* configure your TTS vendor */ )Set credential.mode to byok and credential.fields.api_key to a reference to the key. In the following example, the key is stored as groq in the support_prod set:
"llm": {
"credential": {
"mode": "byok",
"fields": {
"api_key": "$secrets.support_prod.groq"
}
},
"vendor": "groq",
"style": "openai",
"model": "llama-3.3-70b-versatile",
"url": "https://api.groq.com/openai/v1/chat/completions",
"max_history": 32,
"failure_message": "Please hold on a second."
}Set the system prompt and the greeting with the agent's instructions and greeting, not in the llm block.
Key parameters
The llm block takes the following fields. In the SDKs, the LLM class sets them from its options. Use the Groq class. The SDK sets the style to openai.
vendorstringSet to groq.
credentialobjectSet mode to byok and fields.api_key to a $secrets.<set>.<key> reference to your Groq API key. Create and manage your API key from Groq console.
stylestringSet to openai. Groq uses an OpenAI-compatible API.
modelstringRefer to Groq models for available models.
urlstringUse the completions endpoint.
For complete llm block options, see Create an agent. For advanced configuration options, model capabilities, and detailed parameter descriptions, see the Groq API Documentation.