# Manually control start and end of speech (/en/ai/best-practices/manual-turn-control)

> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).

When your business scenario isn't a good fit for letting the server automatically determine when a user starts or stops speaking, use manual Start of Speech (SoS) and End of Speech (EoS) to have the client explicitly declare the user's turn boundaries. This is useful for AI interviews, interactive quizzes, walkie-talkie style push-to-talk, and other "press to start, tap to submit" interactions. This guide explains how to implement manual control over user speech boundaries by configuring the agent and using the client toolkit.

## Understand the tech

Manual turn control consists of two independent capabilities:

* **Manual SoS**: The client explicitly declares that the user has started speaking.
* **Manual EoS**: The client explicitly declares that the user has finished speaking.

`start_of_speech.mode` and `end_of_speech.mode` are independent settings. Choose which boundary needs manual control based on your use case:

| Business scenario                                                                             | `start_of_speech.mode` | `end_of_speech.mode` | Client call                              |
| --------------------------------------------------------------------------------------------- | ---------------------- | -------------------- | ---------------------------------------- |
| The user can start speaking freely, but must tap "submit answer"                              | `vad`                  | `manual`             | Call only `manualEOS`                    |
| The user taps or holds a button to start speaking, and the server determines when speech ends | `manual`               | `vad`                | Call only `manualSOS`                    |
| Press-and-hold to talk, release to send (walkie-talkie mode)                                  | `manual`               | `manual`             | Call `manualSOS` first, then `manualEOS` |

* If only `end_of_speech.mode` is `manual`, the server still uses VAD to automatically detect when the user starts speaking. Your app only needs to call `manualEOS` when the user taps submit.
* If only `start_of_speech.mode` is `manual`, the server starts accepting audio for the current turn only after your app calls `manualSOS`. End of speech is still detected automatically by VAD, so your app does not need to call `manualEOS`.
* If both `start_of_speech.mode` and `end_of_speech.mode` are `manual`, the server starts accepting audio for the current turn only after your app calls `manualSOS`, and submits the turn only after your app calls `manualEOS`.

## Prerequisites

Before you begin, ensure that you have:

* Completed the basic steps for interacting with an agent. See [Quickstart](/en/ai/get-started/quickstart).
* Integrated Video SDK v4.5.1 or later and followed the [Quickstart](../../realtime-media/video/quickstart) guide to implement basic real-time audio and video features.
* Enabled Signaling for your project in the Agora Console and followed the [Signaling Quickstart](/en/realtime-media/rtm/quickstart) to implement real-time messaging.
* Made sure the RTC engine instance is initialized and the app is logged in to Signaling. The client toolkit does not handle initialization, lifecycle management, authentication, or login for Video SDK or Signaling.

## Implement manual control

Manual turn control requires configuring both the agent and the client toolkit:

1. Configure manual mode when you start the agent.
2. Initialize the client toolkit and subscribe to channel messages.
3. Register handlers for the manual SoS/EoS result events.
4. Call `manualSOS`/`manualEOS` when the user taps start or submit.

### Enable manual turn control when starting the agent

When calling [Start a conversational AI agent](/en/api-reference/api-ref/conversational-ai/join), use the following parameter settings:

* `advanced_features.enable_rtm: true`: Start Signaling (Required)
* `parameters.data_channel: "rtm"`: Enable the RTM data transmission channel (Required)
* `parameters.enable_metrics: true`: Receive agent performance data (Enable on demand)
* `parameters.enable_error_message: true`: Receive agent error events (Enable on demand)

Then set the boundary or boundaries you want to control manually to `"manual"`.

The following `curl` examples show three common configurations:

<Tabs>
  <TabsList>
    <TabsTrigger value="eos">
      Manual EoS only
    </TabsTrigger>

    <TabsTrigger value="sos">
      Manual SoS only
    </TabsTrigger>

    <TabsTrigger value="sos-eos">
      Manual SoS + EoS
    </TabsTrigger>
  </TabsList>

  <TabsContent value="eos">
    If you only need to manually control end of speech, keep automatic SoS and set only `end_of_speech.mode` to `manual`.

    ```shell
    curl --request POST \
      --url https://api.agora.io/api/conversational-ai-agent/v2/projects/<your_app_id>/join \
      --header 'Authorization: agora token="007abcxxxxxxx123"' \
      --data '
    {
      "name": "manual-eos-agent",
      "properties": {
        "channel": "channel_name",
        "token": "token",
        "agent_rtc_uid": "0",
        "remote_rtc_uids": [
          "123"
        ],
        "advanced_features": {
          "enable_rtm": true
        },
        "asr": {
          "language": "en-US"
        },
        "llm": {
          "url": "https://api.xxxx/v1/xxxx",
          "api_key": "xxx",
          "system_messages": [
            {
              "role": "system",
              "content": "You are a helpful chatbot."
            }
          ],
          "greeting_message": "Hello, how can I help you?",
          "failure_message": "Sorry, I am unable to answer that question.",
          "max_history": 10,
          "params": {
            "model": "xxxx"
          }
        },
        "tts": {
          "vendor": "minimax",
          "params": {
            "key": "your-minimax-key",
            "model": "speech-01-turbo",
            "voice_setting": {
              "voice_id": "female-shaonv",
              "speed": 1,
              "vol": 1,
              "pitch": 0,
              "emotion": "happy"
            },
            "audio_setting": {
              "sample_rate": 16000
            }
          }
        },
        "turn_detection": {
          "mode": "default",
          "config": {
            "start_of_speech": {
              "mode": "vad",
              "vad_config": {
                "interrupt_duration_ms": 160,
                "speaking_interrupt_duration_ms": 320,
                "prefix_padding_ms": 800
              }
            },
            "end_of_speech": {
              "mode": "manual"
            }
          }
        },
        "parameters": {
          "data_channel": "rtm"
        }
      }
    }
    '
    ```
  </TabsContent>

  <TabsContent value="sos">
    If you only need to manually control start of speech, set `start_of_speech.mode` to `manual` and keep automatic EoS.

    ```shell
    curl --request POST \
      --url https://api.agora.io/api/conversational-ai-agent/v2/projects/<your_app_id>/join \
      --header 'Authorization: agora token="007abcxxxxxxx123"' \
      --data '
    {
      "name": "manual-sos-agent",
      "properties": {
        "channel": "channel_name",
        "token": "token",
        "agent_rtc_uid": "0",
        "remote_rtc_uids": [
          "123"
        ],
        "advanced_features": {
          "enable_rtm": true
        },
        "asr": {
          "language": "en-US"
        },
        "llm": {
          "url": "https://api.xxxx/v1/xxxx",
          "api_key": "xxx",
          "system_messages": [
            {
              "role": "system",
              "content": "You are a helpful chatbot."
            }
          ],
          "greeting_message": "Hello, how can I help you?",
          "failure_message": "Sorry, I am unable to answer that question.",
          "max_history": 10,
          "params": {
            "model": "xxxx"
          }
        },
        "tts": {
          "vendor": "minimax",
          "params": {
            "key": "your-minimax-key",
            "model": "speech-01-turbo",
            "voice_setting": {
              "voice_id": "female-shaonv",
              "speed": 1,
              "vol": 1,
              "pitch": 0,
              "emotion": "happy"
            },
            "audio_setting": {
              "sample_rate": 16000
            }
          }
        },
        "turn_detection": {
          "mode": "default",
          "config": {
            "start_of_speech": {
              "mode": "manual"
            },
            "end_of_speech": {
              "mode": "vad",
              "vad_config": {
                "silence_duration_ms": 480
              }
            }
          }
        },
        "parameters": {
          "data_channel": "rtm"
        }
      }
    }
    '
    ```
  </TabsContent>

  <TabsContent value="sos-eos">
    If you need a complete push-to-talk experience, set both `start_of_speech.mode` and `end_of_speech.mode` to `manual`.

    ```shell
    curl --request POST \
      --url https://api.agora.io/api/conversational-ai-agent/v2/projects/<your_app_id>/join \
      --header 'Authorization: agora token="007abcxxxxxxx123"' \
      --data '
    {
      "name": "manual-sos-eos-agent",
      "properties": {
        "channel": "channel_name",
        "token": "token",
        "agent_rtc_uid": "0",
        "remote_rtc_uids": [
          "123"
        ],
        "advanced_features": {
          "enable_rtm": true
        },
        "asr": {
          "language": "en-US"
        },
        "llm": {
          "url": "https://api.xxxx/v1/xxxx",
          "api_key": "xxx",
          "system_messages": [
            {
              "role": "system",
              "content": "You are a helpful chatbot."
            }
          ],
          "greeting_message": "Hello, how can I help you?",
          "failure_message": "Sorry, I am unable to answer that question.",
          "max_history": 10,
          "params": {
            "model": "xxxx"
          }
        },
        "tts": {
          "vendor": "minimax",
          "params": {
            "key": "your-minimax-key",
            "model": "speech-01-turbo",
            "voice_setting": {
              "voice_id": "female-shaonv",
              "speed": 1,
              "vol": 1,
              "pitch": 0,
              "emotion": "happy"
            },
            "audio_setting": {
              "sample_rate": 16000
            }
          }
        },
        "turn_detection": {
          "mode": "default",
          "config": {
            "start_of_speech": {
              "mode": "manual"
            },
            "end_of_speech": {
              "mode": "manual"
            }
          }
        },
        "parameters": {
          "data_channel": "rtm"
        }
      }
    }
    '
    ```
  </TabsContent>
</Tabs>

After a successful response, the agent joins the specified RTC channel and is ready to interact with the user.

### Integrate and initialize the client toolkit

<Tabs defaultValue="android" groupId="ai-toolkit-platform">
  <TabsList>
    <TabsTrigger value="android">
      Android
    </TabsTrigger>

    <TabsTrigger value="ios">
      iOS
    </TabsTrigger>

    <TabsTrigger value="web">
      Web
    </TabsTrigger>
  </TabsList>

  <TabsContent value="android">
    1. Add the toolkit to your project using Maven or source code. See [Install the Android toolkit](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#installation). Refer to the [component structure](#component-structure) to understand the role of each file.

    2. Create a configuration object for the RTC engine and Signaling client instances, then use it to initialize the component instance.

       ```kotlin
       val config = ConversationalAIAPIConfig(
           rtcEngine = rtcEngine,
           rtmClient = rtmClient,
           renderMode = TranscriptRenderMode.Word,
           enableLog = true,
           enableRenderModeFallback = true
       )
       val api = ConversationalAIAPIImpl(config)
       ```

    3. Register a handler for the manual SoS/EoS result events, then subscribe to the agent's channel.

       ```kotlin
       api.addHandler(object : IConversationalAIAPIEventHandler {
           override fun onUserManualSosEvent(agentUserId: String, event: UserManualSosEvent) {
               // Handle the manual SoS result. event.payload.success indicates whether the server accepted it.
           }

           override fun onUserManualEosEvent(agentUserId: String, event: UserManualEosEvent) {
               // Handle the manual EoS result.
           }

           override fun onAgentManualEosEvent(agentUserId: String, event: AgentManualEosEvent) {
               // Handle the server-triggered automatic EoS notification.
           }
       })

       api.subscribeMessage("channelName") { error ->
           if (error != null) {
               // Handle subscription failure.
           }
       }
       ```
  </TabsContent>

  <TabsContent value="ios">
    1. Add the toolkit to your project using CocoaPods, Swift Package Manager, or source code. See [Install the iOS toolkit](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#installation). Refer to the [component structure](#component-structure) to understand the role of each file.

    2. Create a configuration object for the RTC engine and Signaling client instances, then use it to initialize the component instance.

       ```swift
       let config = ConversationalAIAPIConfig(
           rtcEngine: rtcEngine,
           rtmEngine: rtmEngine,
           renderMode: .words,
           enableLog: true,
           enableRenderModeFallback: true
       )
       convoAIAPI = ConversationalAIAPIImpl(config: config)
       ```

    3. Register a handler for the manual SoS/EoS result events, then subscribe to the agent's channel.

       ```swift
       convoAIAPI.addHandler(handler: self)

       func onUserManualSosEvent(agentUserId: String, event: UserManualSosEvent) {
           // Handle the manual SoS result. event.payload.success indicates whether the server accepted it.
       }

       func onUserManualEosEvent(agentUserId: String, event: UserManualEosEvent) {
           // Handle the manual EoS result.
       }

       func onAgentManualEosEvent(agentUserId: String, event: AgentManualEosEvent) {
           // Handle the server-triggered automatic EoS notification.
       }

       convoAIAPI.subscribeMessage(channelName: channelName) { error in
           if let error = error {
               // Handle subscription failure.
           }
       }
       ```
  </TabsContent>

  <TabsContent value="web">
    1. Add the toolkit to your project using a package manager or source code. See [Install the Web toolkit](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#installation). Refer to the [component structure](#component-structure) to understand the role of each file.

    2. Create a configuration object for the RTC engine and Signaling client instances, then use it to initialize the component instance.

       ```ts
       const config: IConversationalAIAPIConfig = {
         rtcEngine,
         rtmEngine,
         renderMode: ETranscriptHelperMode.WORD,
         enableLog: true,
         enableRenderModeFallback: true,
       }

       const conversationalAIAPI = await ConversationalAIAPI.init(config)
       ```

    3. Register a handler for the manual SoS/EoS result events, then subscribe to the agent's channel.

       ```ts
       conversationalAIAPI.on(EConversationalAIAPIEvents.USER_MANUAL_SOS_RESULT, (agentUserId, event) => {
         // Handle the manual SoS result. event.payload.success indicates whether the server accepted it.
       })

       conversationalAIAPI.on(EConversationalAIAPIEvents.USER_MANUAL_EOS_RESULT, (agentUserId, event) => {
         // Handle the manual EoS result.
       })

       conversationalAIAPI.on(EConversationalAIAPIEvents.AGENT_MANUAL_EOS_RESULT, (agentUserId, event) => {
         // Handle the server-triggered automatic EoS notification.
       })

       conversationalAIAPI.subscribeMessage(channelName)
       ```
  </TabsContent>
</Tabs>

<CalloutContainer type="info">
  <CalloutTitle>
    Info
  </CalloutTitle>

  <CalloutDescription>
    These handlers receive the server's processing result for your manual SoS/EoS requests. Use the `event`, `requestId`, and error information to implement your own business logic. See the [client toolkit API reference](#toolkit) for details.
  </CalloutDescription>
</CalloutContainer>

### Send manual SoS when the user starts speaking

Call `manualSOS` only when `start_of_speech.mode` is `"manual"`. If you use automatic SoS through VAD, skip this step.

<Tabs defaultValue="android" groupId="ai-toolkit-platform">
  <TabsList>
    <TabsTrigger value="android">
      Android
    </TabsTrigger>

    <TabsTrigger value="ios">
      iOS
    </TabsTrigger>

    <TabsTrigger value="web">
      Web
    </TabsTrigger>
  </TabsList>

  <TabsContent value="android">
    ```kotlin
    api.manualSOS(agentUserId = "agentUserId") { requestId, error ->
        if (error != null) {
            Log.e("ManualSOS", "Send failed: ${error.errorMessage}, requestId=$requestId")
        } else {
            Log.i("ManualSOS", "SoS sent, requestId=$requestId")
        }
    }
    ```
  </TabsContent>

  <TabsContent value="ios">
    ```swift
    convoAIAPI.manualSOS(agentUserId: agentUid) { requestId, error in
        if let error = error {
            print("Send failed: \(error.message), requestId=\(requestId)")
        } else {
            print("SoS sent, requestId=\(requestId)")
        }
    }
    ```
  </TabsContent>

  <TabsContent value="web">
    ```ts
    const requestId = await conversationalAIAPI.manualSOS(agentUserId)
    ```
  </TabsContent>
</Tabs>

Once manual SoS takes effect, the server starts counting subsequent audio toward the current user turn. Audio that arrives before this point is not counted.

### Send manual EoS when the user finishes speaking

Call `manualEOS` only when `end_of_speech.mode` is `"manual"`. If you use automatic EoS through VAD, skip this step.

<Tabs defaultValue="android" groupId="ai-toolkit-platform">
  <TabsList>
    <TabsTrigger value="android">
      Android
    </TabsTrigger>

    <TabsTrigger value="ios">
      iOS
    </TabsTrigger>

    <TabsTrigger value="web">
      Web
    </TabsTrigger>
  </TabsList>

  <TabsContent value="android">
    ```kotlin
    api.manualEOS(agentUserId = "agentUserId") { requestId, error ->
        if (error != null) {
            Log.e("ManualEOS", "Send failed: ${error.errorMessage}, requestId=$requestId")
        } else {
            Log.i("ManualEOS", "EoS sent, requestId=$requestId")
        }
    }
    ```
  </TabsContent>

  <TabsContent value="ios">
    ```swift
    convoAIAPI.manualEOS(agentUserId: agentUid) { requestId, error in
        if let error = error {
            print("Send failed: \(error.message), requestId=\(requestId)")
        } else {
            print("EoS sent, requestId=\(requestId)")
        }
    }
    ```
  </TabsContent>

  <TabsContent value="web">
    ```ts
    const requestId = await conversationalAIAPI.manualEOS(agentUserId)
    ```
  </TabsContent>
</Tabs>

Once manual EoS takes effect, the agent submits the turn for ASR and LLM inference as usual. This signal only marks the end of the user's current speech segment — it does not immediately end the entire turn.

### Destroy the component instance

When the agent interaction ends, destroy the component instance to release all resources.

<Tabs defaultValue="android" groupId="ai-toolkit-platform">
  <TabsList>
    <TabsTrigger value="android">
      Android
    </TabsTrigger>

    <TabsTrigger value="ios">
      iOS
    </TabsTrigger>

    <TabsTrigger value="web">
      Web
    </TabsTrigger>
  </TabsList>

  <TabsContent value="android">
    ```kotlin
    api.destroy()
    ```
  </TabsContent>

  <TabsContent value="ios">
    ```swift
    convoAIAPI.destroy()
    ```
  </TabsContent>

  <TabsContent value="web">
    ```ts
    conversationalAIAPI.destroy()
    ```
  </TabsContent>
</Tabs>

## Reference

### API reference

#### RESTful API

* [Start a conversational AI agent](/en/api-reference/api-ref/conversational-ai/join)

#### Toolkit

<Tabs defaultValue="android" groupId="ai-toolkit-platform">
  <TabsList>
    <TabsTrigger value="android">
      Android
    </TabsTrigger>

    <TabsTrigger value="ios">
      iOS
    </TabsTrigger>

    <TabsTrigger value="web">
      Web
    </TabsTrigger>
  </TabsList>

  <TabsContent value="android">
    * [`addHandler`](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#addhandler)
    * [`subscribeMessage`](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#subscribemessage)
    * [`manualSOS`](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#manualsos)
    * [`manualEOS`](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#manualeos)
    * [`destroy`](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#destroy)
    * [`onUserManualSosEvent`](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#onusermanualsosevent)
    * [`onUserManualEosEvent`](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#onusermanualeosevent)
    * [`onAgentManualEosEvent`](/en/api-reference/api-ref/conversational-ai/client-toolkit/android#onagentmanualeosevent)
  </TabsContent>

  <TabsContent value="ios">
    * [`addHandler`](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#addhandler)
    * [`subscribeMessage`](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#subscribemessage)
    * [`manualSOS`](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#manualsos)
    * [`manualEOS`](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#manualeos)
    * [`destroy`](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#destroy)
    * [`onUserManualSosEvent`](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#onusermanualsosevent)
    * [`onUserManualEosEvent`](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#onusermanualeosevent)
    * [`onAgentManualEosEvent`](/en/api-reference/api-ref/conversational-ai/client-toolkit/ios#onagentmanualeosevent)
  </TabsContent>

  <TabsContent value="web">
    * [`subscribeMessage`](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#subscribemessage)
    * [`manualSOS`](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#manualsos)
    * [`manualEOS`](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#manualeos)
    * [`destroy`](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#destroy)
    * [`EConversationalAIAPIEvents`](/en/api-reference/api-ref/conversational-ai/client-toolkit/web#econversationalaiapievents)
  </TabsContent>
</Tabs>