Cartesia

Updated

Integrate Cartesia TTS into Conversational AI Engine.

Cartesia provides ultra-fast, low-latency text-to-speech with real-time streaming capabilities, optimized for interactive conversational AI applications.

Sample configuration

The following examples show how to configure Cartesia TTS when starting a conversational AI agent.

from agora_agent import Agent
from agora_agent.agentkit.vendors import CartesiaTTS

# client is your configured Agora client
agent = (
    Agent(client)
    .with_stt(...)  # configure your STT vendor
    .with_llm(...)  # configure your LLM vendor
    .with_tts(CartesiaTTS(
        api_key='your-cartesia-key',  # omit api_key and set credential_mode='managed' to use Agora-managed mode
        model_id='sonic-2',
        voice_id='<voice_id>',
    ))
)
import { Agent, CartesiaTTS } from 'agora-agents';

// client is your configured Agora client
const agent = new Agent({ client })
  .withStt(/* configure your STT vendor */)
  .withLlm(/* configure your LLM vendor */)
  .withTts(new CartesiaTTS({
    apiKey: 'your-cartesia-key', // omit apiKey and set credentialMode: CredentialMode.Managed to use Agora-managed mode
    modelId: 'sonic-2',
    voiceId: '<voice_id>',
  }));
import (
    "github.com/AgoraIO/agora-agents-go/v2/agentkit"
    "github.com/AgoraIO/agora-agents-go/v2/agentkit/vendors"
)

// client is your configured Agora client
agent := agentkit.NewAgent(client).WithStt(/* configure your STT vendor */).
  WithLlm(/* configure your LLM vendor */).
  WithTts(
    vendors.NewCartesiaTTS(vendors.CartesiaTTSOptions{
        APIKey:  "your-cartesia-key", // omit APIKey and set CredentialMode: agentkit.CredentialModeManaged to use Agora-managed mode
        ModelID: "sonic-2",
        VoiceID: "<voice_id>",
    }),
)
  • Managed mode

    To use Cartesia TTS with Agora-managed credentials, set credential_mode to "managed" in the tts block. When using managed mode, params.api_key is not required. Set params.model_id to sonic-3.5 or sonic-3.6. You can still configure settings such as voice, output_format, and language. For more information, see Use managed mode.

    "tts": {
      "credential_mode": "managed",
      "vendor": "cartesia",
      "params": {
        "model_id": "sonic-3.6",
        "voice": {
          "mode": "id",
          "id": "<voice_id>"
        },
        "output_format": {
          "container": "raw",
          "sample_rate": 16000
        },
        "language": "en"
      }
    }
  • BYOK

    "tts": {
      "vendor": "cartesia",
      "params": {
        "api_key": "<your_cartesia_key>",
        "model_id": "sonic-2",
        "base_url": "wss://api.cartesia.ai",
        "voice": {
          "mode": "id",
          "id": "<voice_id>"
        },
        "output_format": {
          "container": "raw",
          "sample_rate": 16000
        },
        "language": "en"
      }
    }

Key parameters

The following parameters are required when using your own API key (BYOK). When using managed mode, api_key is not required. For more information, see Use managed mode.

params required
ParameterDescription
api_keystring
optional

The Cartesia API key used to authenticate requests. Required when using your own API key (BYOK). Get your API key from the Cartesia Console.

model_idstring
required

The identifier of the TTS model to use. For example, sonic-2. For managed mode, use sonic-3.5 or sonic-3.6.

base_urlstring
optional

The WebSocket URL for the Cartesia streaming API. For example, wss://api.cartesia.ai.

voiceobject
required

Voice configuration object.

  • mode: string

    Voice selection mode. Use id to select by voice identifier.

  • id: string

    The identifier of the selected voice for speech synthesis.

output_formatobject
optional

Audio output format configuration

  • Optional container?: string

    Audio container format for the output stream.

  • Optional sample_rate?: number
    Default value
    16000
    Possible values
    • 8000
    • 16000
    • 22050
    • 24000
    • 44100
    • 48000

    Audio sampling rate in Hz

languagestring
optional

Target language for speech synthesis.

The table extends beyond the current width. Scroll horizontally to view more columns.

Caution

The parameters listed on this page are validated for use with Conversational AI Engine. Required parameters must be provided as documented. Any additional parameters are passed through directly to the underlying vendor without validation. For a full list of supported options, refer to the Cartesia TTS documentation.

This integration is fully supported for use with Conversational AI Engine. While it has completed functional validation, it is newer to the platform, and additional provider-specific edge cases may be identified as usage scales across a broader range of applications and workloads.