Manage recorded files

Updated

Know the naming conventions of the recorded files, how to parse the information in the M3U8 file, and when slicing occurs

Agora Cloud Recording generates HLS files (M3U8 and TS/M4S), DASH files (MPD and WebM), and MP4 and MP3 files. To process these recorded files, such as merging audio and video, converting file formats, or synchronizing playback with other streams, it is essential to understand the following key aspects of recorded files:

  • The naming convention
  • How to parse information from an M3U8 file
  • The conditions under which file slicing occurs

The individual recording content on this page applies to the current version of individual recording. If you use legacy individual recording, where the mode path parameter in the request URL is individual, see Legacy recorded files.

Naming conventions

The naming conventions of recorded files differ by recording mode.

Since the filename does not support special characters, any special characters in cname, including !, @, #, $, %, ^, &, *, (, ), +, _, ., =, [, ], {, }, ~, |, ,, ;, :, ?, <, and >, are replaced with - in the filename.

Individual recording

The recording service generates separate audio and video files for each user ID. The naming conventions depend on the video codec of the source stream.

H.264 or H.265 source streams

  • HLS M3U8 file: <sid>_<cname>__uid_s_<uid>__uid_e_<codec>.m3u8
  • HLS TS slice file: <sid>_<cname>__uid_s_<uid>__uid_e_<utc>_<codec>.ts
  • MP4 file: <sid>_<cname>__uid_s_<uid>__uid_e_<index>_<codec>.mp4
  • Screenshot file: <sid>_<cname>__uid_s_<uid>__uid_e_video_<utc>.jpg

If the resolution changes during the recording, the recording service starts a new MP4 file. The <index> field shows the sequence number of each MP4 file.

VP8 or VP9 source streams

  • DASH MPD file: <sid>_<cname>__uid_s_<uid>__uid_e_<codec>.mpd
  • DASH WebM media slice file: <sid>_<cname>__uid_s_<uid>__uid_e_<stream index>_<segment index>_<codec>.webm
  • DASH WebM initialization file: <sid>_<cname>__uid_s_<uid>__uid_e_<stream index>_<utc>_<codec>_init.webm
  • Screenshot file: <sid>_<cname>__uid_s_<uid>__uid_e_video_<utc>.jpg

MP4 files are not supported.

AV1 source streams

  • HLS M3U8 file: <sid>_<cname>__uid_s_<uid>__uid_e_<codec>.m3u8
  • HLS M4S slice file: <sid>_<cname>__uid_s_<uid>__uid_e_<utc>_av1.m4s. AV1 uses M4S slices and does not generate TS files.
  • HLS M4S initialization file: <sid>_<cname>__uid_s_<uid>__uid_e_<utc>_av1_init.m4s
  • Screenshot file: <sid>_<cname>__uid_s_<uid>__uid_e_video_<utc>.jpg

MP4 files are not supported.

Audio-only source streams

Audio-only individual recording uses AAC encoding by default and generates HLS files with the following names:

  • HLS M3U8 file: <sid>_<cname>__uid_s_<uid>__uid_e_aac.m3u8
  • HLS TS slice file: <sid>_<cname>__uid_s_<uid>__uid_e_<utc>_aac.ts

If you set recordingFileConfig.avFileType to ["hls","mp3"] to generate MP3 files, the files have the following names:

  • HLS M3U8 file: <sid>_<cname>__uid_s_<uid>__uid_e_mp3.m3u8
  • HLS TS slice file: <sid>_<cname>__uid_s_<uid>__uid_e_<utc>_mp3.ts
  • MP3 file: <sid>_<cname>__uid_s_<uid>__uid_e_<index>_mp3.mp3

Naming fields

The single and double underscores in the file names are part of the naming conventions and cannot be omitted. In your business logic, use the complete file names returned in the cloud_recording_file_infos notification whenever possible.

Where:

  • <sid> is the ID of the recording task.
  • <cname> is the channel name.
  • <uid> is the user ID being recorded.
  • <utc> is the UTC time when the slice file starts. The time zone is UTC+0, and the timestamp consists of the year, month, day, hour, minute, second, and millisecond. When utc is 20190611073246073, for example, the slice file starts at 07:32
    .073 a.m., June 11, 2019.
  • <index> is the index number of the MP4 or MP3 file. The index number of the first file is 0. For MP4 files, the recording service generates an additional file when the length of the current file reaches approximately three hours or the size of the file exceeds approximately 2 GB.
  • <stream index> is the index of the WebM stream.
  • <segment index> is the index of the WebM slice.
  • <codec> is the audio or video codec, such as h264, h265, av1, aac, mp3, vp8, or vp9.
  • video is a fixed field in screenshot file names.
  • _init.webm is the fixed suffix of DASH WebM initialization files.

For example, in the file name:

13a4602f55448e83be0b7aaacdda1d22_test_yb__uid_s_222__uid_e_20260807065702767_h264.ts

13a4602f55448e83be0b7aaacdda1d22 is the ID of the recording task, test_yb is the channel name, 222 is the user ID being recorded, 20260807065702767 is the UTC time, and h264 is the video codec.

Composite recording

In composite recording mode, the naming conventions are as follows:

  • M3U8 files: <sid>_<cname>.m3u8
  • TS files: <sid>_<cname>_<utc>.ts
  • MP4 files: <sid>_<cname>_<index>.mp4

Where:

  • <sid> is the recording ID.
  • <cname> is the channel name.
  • <utc> is the starting time (UTC) of the TS file. The time zone is UTC+0. The timestamp consists of the year, month, day, hour, minute, second, and millisecond. For example, if <utc> is 20190611073246073, the starting time of the TS file is 07:32
    .073 a.m., June 11, 2019.
  • <index> is the index number of the MP4 file. The index number of the first MP4 file is 0. The recording service automatically generates an additional MP4 file when the length of the current file reaches approximately three hours or the size of the file exceeds approximately 2 GB.

To get MP4 files, set avFileType as ["hls","mp4"].

Web page recording

In web page recording mode, the naming conventions are as follows:

  • M3U8 files: <sid>_<cname>.m3u8
  • TS files: <sid>_<cname>_<utc>.ts
  • MP4 file: <sid>_<cname>_<index>.mp4

Where:

  • <sid> is the recording ID.
  • <cname> is the value of cname you use in the acquire method.
  • <utc> is the starting time (UTC) of the TS file. The time zone is UTC+0. The timestamp consists of the year, month, day, hour, minute, second, and millisecond. For example, if <utc> is 20190611073246073, the starting time of the TS file is 07:32
    .073 a.m., June 11, 2019.
  • <index> is the index number of the MP4 file. The index number of the first MP4 file is 0. The recording service automatically generates a new MP4 file when the length of the current file reaches about three hours or the size of the file exceeds about 2 GB.

To generate an MP4 file, add mp4 to avFileType.

Recorded files in abnormal conditions

Upload to cloud storage failed

In individual recording, if serverResponse.uploadingStatus in the stop response is stranded, some recorded files have not yet been uploaded to your third-party cloud storage. The recording service keeps retrying the upload automatically until it succeeds, and the files are not lost. Wait until the upload completes before you play or process files that depend on the complete recording.

If Agora Cloud Recording fails to upload the recorded files to the third-party cloud storage, it transfers the files to the third-party cloud storage from Agora Fallback Storage. In order not to overwrite the latest files, the transferred M3U8 file is appended with _<tick>_<index>.m3u8, where:

  • <tick> is related to the time when the index file is generated.
  • <index> is the index of the M3U8 file. The first version to be transferred has an index of 0. The higher the index, the newer the version of the file.

For example, in the following file name, the 3 at the end indicates that this is the fourth version of the M3U8 file:

13a4602f55448e83be0b7aaacdda1d22_test_yb__uid_s_222__uid_e_h264_22194679897_3.m3u8

When you find transferred M3U8 file(s) in the third-party cloud storage, compare the M3U8 file with the highest index to the file without a suffix, and choose the larger file.

The cloud recording service appends this suffix only to M3U8 file names.

Agora Fallback Storage is enabled by default to prevent loss of recorded files caused by third-party cloud storage failures or network congestion. Agora automatically deletes recorded files stored in Agora Fallback Storage after 48 hours. To disable Agora Fallback Storage, contact support@agora.io.

Server disconnected or process killed

When a cloud recording server is disconnected or the process killed, the cloud recording service enables the high availability mechanism, where the fault processing center automatically switches to a new server within 90 seconds to resume the service. Each time the service enables the high availability mechanism, it creates a new M3U8 file, which contains the index information of the recorded slice files from the time when the service resumes. The file name is prefixed with bak<n>, where n stands for the number of times the mechanism is enabled in a recording, starting with 0.

For example, in the filename bak0_13a4602f55448e83be0b7aaacdda1d22_test_yb__uid_s_222__uid_e_h264.m3u8, bak0 indicates that this file is generated after enabling the high availability mechanism for the first time.

After the cloud recording service enables the high availability mechanism, the names of the recorded TS/WebM files and the MP4 files are also prefixed with bak<n>.

To merge the original M3U8 file and the new M3U8 file generated after enabling the high availability mechanism into an MP4 file, use the transcoding script for file merging as follows (only applicable to merged recording files):

  1. Ensure that the server executing the transcoding script uses one of the following systems:

    • Ubuntu 14.04+ x64
    • CentOS 7.0+ x64
    • Debian 8.0+
  2. Enter the transcoder and ffmpeg paths in the terminal.

  3. Execute the following command:

    ./ha_transcoder.exe -inputPath "<YourinputPath>" -ignoreNotExist -concatM3u8 -concatStrategy 0
    ParameterDescriptionRequired
    -concatM3u8Merge the original M3U8 file and the new M3U8 file generated after enabling the high availability mechanism into one MP4 file. If this parameter is not set, the original M3U8 file and the new M3U8 file will be converted into MP4 files respectively.no
    -concatStrategy

    Merge strategy, this parameter must be consistent with -concatM3u8. Can be set to:

    • 0: (Default) If there is an overlap between two M3U8 files, the overlapping part of the original M3U8 file will be cut off; if there is a time interval between the two M3U8 files, the black frame of the corresponding time interval will be added.
    • 1: Merge two M3U8 files regardless of the overlap or time interval.
    no
    -ignoreNotExistMissing TS files are ignored and black frames are used instead. If this parameter is not set, the absence of any TS file will cause the conversion to fail.no
    -inputPathInput path, used to specify the absolute storage path of the files to be merged.yes
    -outputPathOutput path, used to specify the absolute storage path of the merged files. Defaults to the same as the input path.no

Estimate file size

In an individual recording, the size of a recorded file depends on the audio and video bitrates of the media source and the duration of the recording. For example, if the audio bitrate is 48 Kbps, the video bitrate is 500 Kbps, and the recording lasts for 30 minutes (1800 seconds), the size of the recorded file is approximately (48 Kbps + 500 Kbps) * 1800 s = 986.4 Mbit, or 123.3 MB.

In a composite recording, the size of a recorded file depends on the audio and video bitrates of the transcoding configurations and the duration of the recording. For example, if you set the audioProfile to 1 (sets the audio bitrate to 128 Kbps), bitrate to 800 (sets the video bitrate to 800 Kbps), and the recording lasts for 30 minutes (1800 seconds), the size of the recorded file is approximately (128 Kbps + 800 Kbps) * 1800 s = 1670.4 Mbit, or 208.8 MB.

M3U8 file

The M3U8 file contains the file names of several slice files and descriptive symbols. The M3U8 file generated by Agora Cloud Recording has three descriptive symbols:

  • #EXT-X-AGORA-TRACK-EVENT:EVENT=<event>,TRACK_TYPE=<type>,TIME=<utc>: The first slice file after a stream starts or restarts after an interruption has this descriptive symbol, which describes the state of the stream.

    • EVENT: Name of the event. Currently, EVENT can only be START, which means that the stream has started or restarted after an interruption.
    • TRACK_TYPE: The content of the slice file, AUDIO or VIDEO.
    • TIME: The time when the state of the stream changes. TIME is in UTC and the time zone is UTC+0.
  • #EXT-X-AGORA-ROTATE:WIDTH=<width>,HEIGHT=<height>,ROTATE=<rotate>,TIME=<utc>: The first slice file after a video rotates has this descriptive symbol, which describes the details of the rotation. A slice file may have several such symbols.

    • WIDTH: Width of the video.
    • HEIGHT: Height of the video.
    • ROTATE: The degrees by which the video rotates anticlockwise. ROTATE can only be 0, 90, 180, or 270.
    • TIME: The time when the video rotates. TIME is in UTC and the time zone is UTC+0.
  • #EXTINF:<length>: Length of the slice file in seconds.

For example:

#EXT-X-AGORA-ROTATE:WIDTH=640,HEIGHT=480,ROTATE=90,TIME=20190920125142485
#EXT-X-AGORA-TRACK-EVENT:EVENT=START,TRACK_TYPE=VIDEO,TIME=20190920125142485
#EXTINF:6.332000
sid713476478245_cnameagora__uid_s_123__uid_e_20190920125142485_h264.ts

The sample M3U8 file above contains the file name of a TS file and three descriptive symbols, which indicates that the TS file is the first slice file after the video stream starts or restarts after an interruption, the video rotates by 90 degrees anticlockwise, and the TS file lasts for 6.332 seconds.

If you face compatibility issues in some M3U8 players due to the missing comma after #EXTINF:<length>, set privateParams in recordingConfig in the start request as follows: "recordingConfig": {"privateParams":"{\"correctEXTINF\":true}", ...}.

Sample M3U8 file for audio

The following sample M3U8 file indexes the audio slices of one user ID in individual recording. It uses the default AAC encoding, and its TS slice file names end with _aac.ts. If you generate MP3 files, the TS slice file names end with _mp3.ts.

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:18
#EXT-X-DISCONTINUITY
#EXT-X-AGORA-TRACK-EVENT:EVENT=START,TRACK_TYPE=AUDIO,TIME=20190920125142289
#EXTINF:15.019000
sid713476478245_cnameagora__uid_s_123__uid_e_20190920125142289_aac.ts
#EXTINF:15.019000
sid713476478245_cnameagora__uid_s_123__uid_e_20190920125157307_aac.ts
#EXTINF:15.019000
sid713476478245_cnameagora__uid_s_123__uid_e_20190920125212326_aac.ts
#EXTINF:15.019000
sid713476478245_cnameagora__uid_s_123__uid_e_20190920125227345_aac.ts
#EXTINF:12.523000
sid713476478245_cnameagora__uid_s_123__uid_e_20190920125242363_aac.ts
#EXT-X-ENDLIST

Sample M3U8 file for video

The following sample M3U8 file indexes the video slices of one user ID in individual recording. The #EXT-X-AGORA-ROTATE symbols show where the video resolution changes:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:34
#EXT-X-DISCONTINUITY
#EXT-X-AGORA-ROTATE:WIDTH=640,HEIGHT=480,ROTATE=0,TIME=20190920125142485
#EXT-X-AGORA-TRACK-EVENT:EVENT=START,TRACK_TYPE=VIDEO,TIME=20190920125142485
#EXTINF:6.332000
sid713476478245_cnameagora__uid_s_123__uid_e_20190920125142485_h264.ts
#EXT-X-AGORA-ROTATE:WIDTH=1280,HEIGHT=720,ROTATE=0,TIME=20190920125149174
#EXT-X-DISCONTINUITY
#EXTINF:17.442000
sid713476478245_cnameagora__uid_s_123__uid_e_20190920125149174_h264.ts
#EXT-X-DISCONTINUITY
#EXT-X-AGORA-ROTATE:WIDTH=640,HEIGHT=480,ROTATE=0,TIME=20190920125206616
#EXTINF:33.326000
sid713476478245_cnameagora__uid_s_123__uid_e_20190920125206616_h264.ts
#EXT-X-DISCONTINUITY
#EXT-X-AGORA-ROTATE:WIDTH=1280,HEIGHT=720,ROTATE=0,TIME=20190920125239942
#EXTINF:14.815000
sid713476478245_cnameagora__uid_s_123__uid_e_20190920125239942_h264.ts
#EXT-X-ENDLIST

Slicing

Slicing splits a continuous audio or video stream into a series of smaller files. The slicing rules differ between individual recording and the other recording modes.

Slicing in individual recording

The recording service slices HLS, DASH, MP4, and MP3 files under different conditions.

HLS and DASH files

When the video codec of the source stream for a user ID changes, the recording service generates a separate index file for each codec. H.264, H.265, and AV1 source streams use M3U8 index files, and VP8 and VP9 source streams use MPD index files. Each index file references only the slices of its own codec.

For example, if the source stream uses VP8 when the recording starts, the recording service generates an MPD file. If the source stream then switches to H.264, the recording service generates an M3U8 file.

Slicing of audio and video files occurs when any of the following conditions is met:

  • When an I frame appears and the slice file reaches 15 seconds.
  • The codec changes.
  • The width or height of the video changes.
  • The rotation of the video changes.
  • The video stream is interrupted.

MP4 files

The recording service generates a new MP4 file when any of the following conditions is met:

  • The codec changes, for example, from H.265 to H.264.
  • The width or height of the video changes.
  • The length of the current MP4 file exceeds approximately three hours.
  • The size of the current MP4 file exceeds approximately 2 GB.

MP3 files

The recording service generates a new MP3 file when the length of the current MP3 file exceeds the configured slice duration. You can set the slice duration from 30 to 120 minutes. The default is 30 minutes.

Slicing in composite and web page recording

The recording service slices HLS files by duration and creates new MP4 files by duration or file size.

HLS files

The M3U8 index file is not sliced. Audio and video media files are sliced by duration. The default slice duration is 15 seconds. If you set sliceDuration for HLS, the configured value applies.

MP4 files

The recording service generates a new MP4 file when any of the following conditions is met:

  • The length of the current MP4 file exceeds the default of three hours. In web page recording, if you set maxVideoDuration, the configured value applies.
  • The size of the current MP4 file exceeds approximately 2 GB.