Groq

Updated

Integrate Groq LLM into Conversational AI Engine.

Groq provides state-of-the-art language models optimized for different use cases.

Sample configuration

The following example shows how to configure Groq LLM when starting a conversational AI agent.

from agora_agent import Agent, Groq

# client is your configured Agora client
agent = (
    Agent(client)
    .with_stt(...)  # configure your STT vendor
    .with_llm(Groq(
        api_key='your-groq-key',
        base_url='https://api.groq.com/openai/v1/chat/completions',
        model='llama-3.3-70b-versatile',
        system_messages=[{'role': 'system', 'content': 'You are a helpful chatbot.'}],
        greeting_message='Hello, how can I assist you?',
        failure_message='Please hold on a second.',
        max_history=32,
    ))
    .with_tts(...)  # configure your TTS vendor
)
import { Agent, Groq } from 'agora-agents';

// client is your configured Agora client
const agent = new Agent({ client })
  .withStt(/* configure your STT vendor */)
  .withLlm(new Groq({
    apiKey: 'your-groq-key',
    url: 'https://api.groq.com/openai/v1/chat/completions',
    model: 'llama-3.3-70b-versatile',
    systemMessages: [{ role: 'system', content: 'You are a helpful chatbot.' }],
    greetingMessage: 'Hello, how can I assist you?',
    failureMessage: 'Please hold on a second.',
    maxHistory: 32,
  }))
  .withTts(/* configure your TTS vendor */);
import "github.com/AgoraIO/agora-agents-go/v2/agentkit/vendors"

// client is your configured Agora client
agent := agentkit.NewAgent(client).WithStt(/* configure your STT vendor */).
  WithLlm(
    vendors.NewGroq(vendors.GroqOptions{
        APIKey:  "your-groq-key",
        BaseURL: "https://api.groq.com/openai/v1/chat/completions",
        Model:   "llama-3.3-70b-versatile",
        SystemMessages: []map[string]interface{}{
            {"role": "system", "content": "You are a helpful chatbot."},
        },
        GreetingMessage: "Hello, how can I assist you?",
        FailureMessage:  "Please hold on a second.",
        MaxHistory:      agora.Int(32),
    }),
).WithTts(/* configure your TTS vendor */)

Use the following llm configuration in your request:

"llm": {
    "url": "https://api.groq.com/openai/v1/chat/completions",
    "api_key": "<your_llm_key>",
    "system_messages": [
      {
        "role": "system",
        "content": "You are a helpful chatbot."
      }
    ],
    "max_history": 32,
    "greeting_message": "Hello, how can I assist you?",
    "failure_message": "Please hold on a second.",
    "params": {
      "model": "llama-3.3-70b-versatile"
    }
}

Key parameters

llmrequired
api_keystring
required

Create and manage your API key from Groq console.

urlstring
required

Use the completions endpoint.

paramsobject
required
modelstring
required

Refer to Groq models for available models.

For advanced configuration options, model capabilities, and detailed parameter descriptions, see the Groq API Documentation.