Skip to main content
Conversational AI Engine: Create advanced voice apps by combining Agora's real-time audio with powerful conversational AI.

Run the R1 demo

Agora has partnered with Wi-Fi chip manufacturers to release the R1 open-source development kit based on Broadcom's BK7258 chip. To try the sample or review the source code, view the GitHub repository.

Convo AI Device Kit sample

Follow this guide to run the demo project and get started quickly with the Convo AI Device Kit.

Prerequisites​

Before you begin, ensure you have the following:

  • Python 3.7 or later for running the server-side components
  • R1 development board: Contact Agora Sales to purchase the R1 development board jointly developed by Agora and Broadcom. Refer to the BK official documentation to set up your development environment.
  • Android device for installing and testing the mobile app
  • Third-party AI services: API keys and service endpoints for your chosen Large Language Model (LLM) and Text-to-Speech (TTS) providers
  • Agora project credentials: An Agora project with Conversational AI Engine enabled, including:
    • App ID
    • App Certificate
    • Customer ID
    • Customer Key
    For setup instructions, see Enable services.

Set up the sample project​

Follow these steps to download and set up the sample project on your local machine.

  1. Clone the Conversational-AI-IOT-Sample repository to your local machine:


    _1
    git clone https://github.com/AgoraIO-Community/Conversational-AI-IOT-Sample.git

  2. Switch to the bk7258/v2.0.1 branch:


    _2
    cd Conversational-AI-IOT-Sample
    _2
    git checkout bk7258/v2.0.1

Configure and run the server​

This section shows you how to install the required Python dependencies and configure the server with your project credentials.

caution

This server-side example is provided solely for demonstration and testing purposes and must not be used in a production environment. For production use, implement a dedicated server-side service that meets your security and scalability requirements.

Install Python dependencies​

Install the required Python packages:


_1
pip install requests flask pyjwt

Configure server settings​

The /server/aiot_server_demo_example/config.json file contains the server configuration parameters. Update this file with your project credentials and service settings.

For detailed parameter descriptions, see Start a conversational AI agent.

Configuration file structure:


_63
{
_63
// Agora project configuration
_63
// Your Agora project App ID
_63
"app_id": "YOUR_AGORA_APP_ID",
_63
// Your Agora project App Certificate
_63
"app_certificate": "YOUR_AGORA_APP_CERTIFICATE",
_63
_63
// Customer authentication
_63
// Customer ID
_63
"customer_key": "YOUR_CUSTOMER_KEY",
_63
// Customer secret
_63
"customer_secret": "YOUR_CUSTOMER_SECRET",
_63
_63
// Automatic Speech Recognition (ASR) configuration
_63
"asr": {
_63
// Recognition language
_63
"language": "en-US"
_63
},
_63
_63
// System parameters
_63
"parameters": {
_63
// RTC audio encoding format, supported formats: "PCMU", "PCMA", "G722", "OPUS", "OPUSFB"
_63
"output_audio_codec": "G722"
_63
},
_63
_63
// Text-to-Speech (TTS) configuration
_63
"tts": {
_63
// TTS service provider
_63
"vendor": "YOUR_TTS_VENDOR",
_63
"params": {
_63
_63
}
_63
},
_63
_63
// Session timeout configuration
_63
// Session timeout in seconds
_63
"idle_timeout": 30,
_63
_63
// Large Language Model (LLM) configuration
_63
"llm": {
_63
// LLM service URL
_63
"url": "YOUR_LLM_API_URL",
_63
"params": {
_63
// Model to use
_63
"model": "YOUR_LLM_MODEL"
_63
},
_63
// LLM service API key
_63
"api_key": "YOUR_LLM_API_KEY",
_63
// System preset messages
_63
"system_messages": [
_63
{
_63
"role": "system",
_63
"content": "You are a helpful chatbot."
_63
}
_63
],
_63
// Maximum history records
_63
"max_history": 10,
_63
// Welcome message
_63
"greeting_message": "Hello, this is your AI assistant. How can I help you?",
_63
// Failure message
_63
"failure_message": "Sorry, I am temporarily unable to answer your question."
_63
}
_63
}

Run the server​

Run the server from the /server/aiot_server_demo_example directory:


_2
cd server/aiot_server_demo_example
_2
python3 main.py

The server runs on port 5001 by default at http://localhost:5001. To use a different port, modify the port configuration in the code.

info

The server provides three REST APIs to support device-side business operations. For details, see the server-side interface documentation.

Configure network access​

The R1 kit includes an Android app in the app folder for configuring device network access through Bluetooth Low Energy (BLE). The app provides a simple interface for connecting your R1 device to Wi-Fi.

This app is built using Agora's Bluetooth configuration library. After installation, the app interface looks like this:

info

Before configuring network access, ensure the following:

  • Bluetooth and location services are enabled on your Android device.
  • The R1 device is in discoverable mode.
  • Your Android device is within 5 meters of the R1 device.
  • Your Android device is connected to a 2.4 GHz Wi-Fi network with internet access. Network configuration fails without internet connectivity.
  • For Android 12 and later, you have granted BLUETOOTH_SCAN and BLUETOOTH_CONNECT permissions to the app.

Connect your device to Wi-Fi​

Follow these steps to connect your R1 device to Wi-Fi:

  1. Grant the required permissions when prompted by the app.

  2. Verify your current Wi-Fi network information.

  3. Enter your Wi-Fi password.

  4. Scan for and select your target Bluetooth device.

  5. Click Connect to establish a connection with the device.

  6. After the connection succeeds, click Configure Network to send the Wi-Fi credentials to the device.

  7. After the network configuration completes, click Disconnect to end the Bluetooth connection.

Android app

Set up network access​

Follow these steps to set up network access:

  1. Request permissions. When you first open the app, grant the required permissions:

    1. Tap Request Permissions. The app requests Bluetooth and location permissions needed for scanning and connecting to Bluetooth devices.
    2. Tap Allow in the system permission dialog.

  2. Verify Wi-Fi permissions. Tap Check Wi-Fi Permissions. The app checks whether it has the necessary Wi-Fi permissions and displays the result.

  3. Get Wi-Fi information. Tap Get Current Wi-Fi Information. The app retrieves and displays your current Wi-Fi network name (SSID) in a card below the button.

  4. Enter Wi-Fi password. Enter your Wi-Fi network password in the password field. The device uses this password to connect to your Wi-Fi network.

  5. Scan for Bluetooth devices:

    1. Tap Start Scan. The app scans for nearby Bluetooth devices. The button text changes to Stop Scanning during the scan.
    2. Scanned devices appear in the list below.
    3. Tap Stop Scanning to end the scan.

  6. Connect to your device. Each device card in the list displays:

    • Device name
    • MAC address
    • Signal strength (RSSI)
    • Connect: Establishes a Bluetooth connection with the device
    • Configure Network: Sends Wi-Fi credentials to the device
    • Disconnect: Ends the Bluetooth connection

    If network configuration fails, reconnect the device or restart the app.

Troubleshooting​

If you encounter issues during network configuration, try the following solutions:

  • Cannot scan for devices: Verify that Bluetooth and location services are enabled on your Android device.
  • Connection fails: Rescan for the device or restart the R1 device.
  • Network configuration fails: Verify that you entered the correct Wi-Fi password.

Run the demo project​

Download and configure the device firmware​

Follow these steps to set up the R1 device firmware with the example project.

  1. Clone the bk_aidk project from GitHub:


    _4
    git clone --recurse-submodules https://github.com/bekencorp/bk_aidk.git -b ai_release/v2.0.1
    _4
    cd bk_aidk
    _4
    git checkout ai_release/v2.0.1.8
    _4
    git submodule update --recursive

  2. Replace the projects directory with the example demo directory. Replace <bk_aidk_path> with your actual path:


    _2
    rm -rf <bk_aidk_path>/projects/
    _2
    cp -r ./projects <bk_aidk_path>/projects

    info

    To avoid conflicts, delete the original projects directory before copying.

  3. Update the server URL in projects/common_components/network_transfer/agora_rtc/agora_config.h to point to your deployed server:


    _1
    #define CONFIG_AGENT_SERVER_URL "http://192.168.1.100:5001"

Replace http://192.168.1.100:5001 with your server address.

Build the firmware​

Build the sample project firmware from the bk_aidk directory:


_2
cd <bk_aidk_path>
_2
make bk7258 PROJECT=beken_genie

Replace <bk_aidk_path> with your actual bk_aidk directory path.

Flash the firmware​

After the firmware builds successfully, flash it to your R1 device. For detailed instructions, see the BK official documentation.

Run the example​

Connect the R1 development board to your computer using a Type-C cable connected to the USB TO UART interface. Follow these steps to test the voice AI functionality:

  1. Configure Wi-Fi (first-time setup): Press and hold the S1 button for 5 seconds to enter network configuration mode. Use the Android app to complete the Wi-Fi setup.

  2. Wake the device: Say "hi, Amino" to wake the device and start a conversation with the AI agent.

  3. End the conversation: Say "bye bye, Amino" to exit the conversation with the AI agent.

  4. Restart from sleep: The device enters deep sleep mode after 3 minutes of inactivity. Press the RST button to restart.

info
  • The USB TO UART interface also charges the battery. The device automatically activates when you connect a battery or data cable.
  • If the programming tool fails to restart the development board automatically, press the RST button to manually restart and restore programming capability.

Reference​

Sample project​

Convo AI Device Kit sample

Server-side interface documentation​

The server provides three RESTful APIs for device operations. This example project does not implement authentication. You can add authentication as needed.

Base URL: https://your-domain.com/

Authentication: None

Get device RTC token​

Retrieves RTC credentials for the device to join a channel.

  • Method: POST

  • Endpoint: /device

  • Request body:


    _4
    {
    _4
    "channel_name": "DEVICE_ID",
    _4
    "uid": DEVICE_USER_ID
    _4
    }

Start conversational AI agent​

Starts the Conversational AI Engine for the specified channel.

  • Method: POST

  • Endpoint: /agent/start

  • Request body:


    _5
    {
    _5
    "channel_name": "DEVICE_ID",
    _5
    "uid": DEVICE_USER_ID,
    _5
    "agent_uid": AGENT_USER_ID
    _5
    }

Stop conversational AI agent​

Stops the Conversational AI Engine.

  • Method: POST

  • Endpoint: /agent/stop

  • Request body:


    _3
    {
    _3
    "agent_id": "AGENT_ID"
    _3
    }