# Smallest AI (/en/ai/platform/models/asr/smallest-ai)

> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).

> Documentation: Conversational AI.
> Documentation version: Preview.
> This document describes only this version. [Version index](/llms/doc-versions-convoai-guides-v2.txt).


Smallest AI provides real-time streaming transcription through its Pulse model. Use it as the ASR model in a cascading pipeline with any supported LLM and TTS vendor.

## Sample configuration

The following example shows how to configure Smallest AI as the agent's ASR when you create or update an agent. For a complete example, see [Create an agent and start a session](/en/ai/platform/get-started/start-stop-agent#create-an-agent).

Smallest AI supports BYOK only. It isn't available in managed mode. Store your Smallest AI API key in a secret set, and refer to it from the agent. For details, see [Use your own provider keys](/en/ai/platform/build/use-your-own-keys).

**Python SDK**

The SDK converts the boolean options to the `"true"` and `"false"` strings that Smallest AI expects.

```python
from agora_agent import Agent, SmallestAISTT, secret_ref

# client is your configured Agora client
agent = (
    Agent(client=client)
    .with_stt(
        SmallestAISTT(
            api_key=secret_ref("support_prod", "smallestai"),
            language="zh",
            url="wss://api.us.smallest.ai/waves/v1/stt/live",
            sample_rate=16000,
            encoding="linear16",
            word_timestamps=True,
            sentence_timestamps=True,
            diarize=True,
            vad_events=True,
            endpointing=True,
            eou_timeout_ms=480,
            format=True,
            finalize_on_words=True,
            max_words="100",
            punctuate=True,
            capitalize=True,
            itn_normalize=True,
            full_transcript=True,
            keywords="Codex:2,Smallest AI:2",
            redact_pii=False,
            redact_pci=False,
        )
    )
    .with_llm(...)  # configure your LLM vendor
    .with_tts(...)  # configure your TTS vendor
)
```

**TypeScript SDK**

The SDK converts the boolean options to the `"true"` and `"false"` strings that Smallest AI expects.

```ts
import { Agent, SmallestAISTT, secretRef } from 'agora-agents';

// client is your configured Agora client
const agent = new Agent({ client })
  .withStt(
    new SmallestAISTT({
      apiKey: secretRef('support_prod', 'smallestai'),
      language: 'zh',
      url: 'wss://api.us.smallest.ai/waves/v1/stt/live',
      sampleRate: 16000,
      encoding: 'linear16',
      wordTimestamps: true,
      sentenceTimestamps: true,
      diarize: true,
      vadEvents: true,
      endpointing: true,
      eouTimeoutMs: 480,
      format: true,
      finalizeOnWords: true,
      maxWords: '100',
      punctuate: true,
      capitalize: true,
      itnNormalize: true,
      fullTranscript: true,
      keywords: 'Codex:2,Smallest AI:2',
      redactPii: false,
      redactPci: false,
    }),
  )
  .withLlm(/* configure your LLM vendor */)
  .withTts(/* configure your TTS vendor */);
```

**Go SDK**

The SDK converts the boolean options to the `"true"` and `"false"` strings that Smallest AI expects.

```go
sampleRate, eouTimeoutMs := 16000, 480

// client is your configured Agora client
agent := agentkit.NewAgent(client).
	WithStt(vendors.NewSmallestAISTT(vendors.SmallestAISTTOptions{
		APIKey:             agentkit.SecretRef("support_prod", "smallestai"),
		Language:           "zh",
		URL:                "wss://api.us.smallest.ai/waves/v1/stt/live",
		SampleRate:         &sampleRate,
		Encoding:           "linear16",
		WordTimestamps:     true,
		SentenceTimestamps: true,
		Diarize:            true,
		VADEvents:          true,
		Endpointing:        true,
		EOUTimeoutMs:       &eouTimeoutMs,
		Format:             true,
		FinalizeOnWords:    true,
		MaxWords:           "100",
		Punctuate:          true,
		Capitalize:         true,
		ITNNormalize:       true,
		FullTranscript:     true,
		Keywords:           "Codex:2,Smallest AI:2",
	})).
	WithLlm( /* configure your LLM vendor */ ).
	WithTts( /* configure your TTS vendor */ )
```

**REST API**

Set `credential.mode` to `byok` and `credential.fields.api_key` to a reference to your Smallest AI API key in a secret set. In the following example, the key is stored as `smallestai` in the `support_prod` set. Only `credential` and `vendor` are required:

```json
"asr": {
  "credential": {
    "mode": "byok",
    "fields": {
      "api_key": "$secrets.support_prod.smallestai"
    }
  },
  "vendor": "smallestai",
  "language": "zh",
  "url": "wss://api.us.smallest.ai/waves/v1/stt/live",
  "params": {
    "sample_rate": 16000,
    "encoding": "linear16",
    "word_timestamps": "true",
    "sentence_timestamps": "true",
    "diarize": "true",
    "vad_events": "true",
    "endpointing": "true",
    "eou_timeout_ms": 480,
    "format": "true",
    "finalize_on_words": "true",
    "max_words": "100",
    "punctuate": "true",
    "capitalize": "true",
    "itn_normalize": "true",
    "full_transcript": "true",
    "keywords": "Codex:2,Smallest AI:2",
    "redact_pii": "false",
    "redact_pci": "false"
  }
}
```

> **Info**
>
> Smallest AI expects its boolean options as the strings `"true"` and `"false"`, as shown in the example.

## Key parameters

The `asr` block takes the following fields. In the SDKs, `SmallestAISTT` sets them from its options, and passes any other Smallest AI parameters from its additional parameters option to `params`.

**asr required**

**`vendor` (string) required**

Set to `smallestai`.

**`credential` (object) required**

How the agent authenticates with Smallest AI. Set `mode` to `byok` and `fields.api_key` to a `$secrets.<set>.<key>` reference to your Smallest AI API key.

**`language` (string) optional**

The language code for speech recognition, for example, `zh`.

**`url` (string) optional**

The Smallest AI streaming ASR WebSocket endpoint, for example, `wss://api.us.smallest.ai/waves/v1/stt/live`. Omit it to use the default endpoint.

**`params` (object) optional**

**`sample_rate` (integer) optional**

Input audio sample rate in Hz. Defaults to `16000`.

**`encoding` (string) optional**

Input audio encoding. Defaults to `linear16`.

**`word_timestamps` (string) optional**

Whether to include word-level timestamps in the transcription results.

**`sentence_timestamps` (string) optional**

Whether to include sentence-level timestamps in the transcription results.

**`diarize` (string) optional**

Whether to enable speaker diarization. Speaker labels are emitted on final transcription results.

**`vad_events` (string) optional**

Whether to emit voice activity detection events.

**`endpointing` (string) optional**

Whether to enable automatic end-of-utterance detection.

**`eou_timeout_ms` (integer) optional**

End-of-utterance timeout in milliseconds. For example, `480`.

**`format` (string) optional**

Whether to enable Smallest AI's transcript formatting. Smallest AI defines the formatting applied.

**`finalize_on_words` (string) optional**

Whether to finalize transcription results based on word count, together with `max_words`.

**`max_words` (string) optional**

Maximum number of words per transcription result, passed as a string. For example, `"100"`.

**`punctuate` (string) optional**

Whether to add punctuation to transcription results.

**`capitalize` (string) optional**

Whether to apply capitalization to transcription results.

**`itn_normalize` (string) optional**

Whether to apply inverse text normalization, which converts spoken forms such as numbers and dates into written form. Smallest AI applies it only to finalized transcripts, before redaction.

**`full_transcript` (string) optional**

Whether to return the full accumulated transcript.

**`keywords` (string) optional**

Keyword boosts that improve recognition accuracy for specific terms, as a comma-separated list in `keyword:weight` format. For example, `Codex:2,Smallest AI:2`.

**`redact_pii` (string) optional**

Whether to redact personally identifiable information from transcription results.

**`redact_pci` (string) optional**

Whether to redact payment card information from transcription results.

> **Caution**
>
> The parameters listed on this page are validated for use with the Conversational AI Platform. Required parameters must be provided as documented. Any additional parameters in `params` are passed through directly to Smallest AI without validation. For a full list of supported options, refer to the [Smallest AI documentation](https://docs.smallest.ai/models/documentation/speech-to-text-pulse/overview).

This integration is fully supported for use with the Conversational AI Platform. While it has completed functional validation, it is newer to the platform, and additional provider-specific edge cases may be identified as usage scales across a broader range of applications and workloads.
