Message moderation mechanism

During message moderation, the content moderation service reviews the specifically designated fields within the message bodies. Different message types have different field names and content structures. Ensure these fields contain only the content to be moderated, as other content may affect moderation accuracy. This guide describes the message content fields within the bodies for each message type.

Following is a sample message structure:

{
  "msg_id": "5I02W-XX-8278a",
  "timestamp": 1403099033211,
  "direction": "outgoing",
  "to": "XXXX",
  "from": "XXXX",
  "chat_type": "chat",
  "payload": {
    "bodies": [
      {
        // message body details 
      }
    ],
    "ext": {
      "key1": "value1"
      // more key-value pairs 
    },
    "from": "XXXX",
    "to": "XXXX"
  }
}

Text messages

For text messages, the content moderation service only reviews the content in the msg field within the message bodies.

Parameter	Type	Description
`msg`	String	Message content.

Example:

{
  "bodies": [
    {
      "msg": "welcome to Agora Chat!",
      "type": "txt"
    }
  ]
}

Image messages

For image messages, the content moderation service downloads files from their URLs for review. The service focuses on the url field within the bodies of these messages.

Parameter	Type	Description
`url`	String	URL address of the image message.

Image message bodies example:

{
  "bodies": [
    {
      "file_length": 128827,
      "filename": "test1.jpg",
      "secret": "DRGM8OZrEeO1vaXXXXXXXXHBeKlIhDp0GCnFu54xOF3M6KLr",
      "size": {
        "height": 1325,
        "width": 746
      },
      "type": "img",
      "url": "https://XXXX/XXXX/chatdemoui/chatfiles/65e54a4a-XXXX-XXXX-b821-ebde7b50cc4b"
    }
  ]
}

Custom messages

info

To use custom message moderation, Enable the moderation feature in Agora Console, then contact technical support to enable custom message moderation.

For custom messages submitted for moderation, the customExts parameter must include a moderation node, as shown in the following example:

{
  "customExts": {
    "moderation": {
      "enable": true,
      "contents": [
        {
          "type": "text",
          "data": "Hello World"
        }
      ]
    }
  }
}

The parameters in the moderation node are described in the following table:

Parameter	Type	Description
`enable`	Boolean	Whether to enable moderation. - Set to `true` to enable moderation. - Set to `false` to disable moderation.
`contents`	Array	Message content.
`type`	String	Message type. Currently, the moderation service supports only text and image messages, so this field can only be set to `text` or `img`.
`data`	String	Content to be moderated. - For text messages: the actual message content. - For image messages: the image URL.

Text messages​

Image messages​

Custom messages​

Text messages

Image messages

Custom messages