# Raw audio processing (/en/realtime-media/voice/build/customize-audio-processing/stream-raw-audio/windows)

> For AI agents: see the complete documentation index at [llms.txt](/llms.txt).

In some use-cases, raw audio captured through the microphone must be processed to enhance the user experience or achieve the desired functionality Voice SDK enables you to pre-process and post-process the captured audio for implementation of custom playback effects.

      
  
      
  
      
  
      
  
      
    This article shows you how to pre-process and post-process collected raw audio data.

    ## Understand the tech [#understand-the-tech-4]

    For use-cases that require self-processing of audio data, Agora Voice SDK provides raw data processing functionality. You can perform pre-processing to modify the captured audio signal before sending the data to the encoder, or post-process data to modify the received audio signal after sending the data to the decoder.

    To implement processing of raw audio data in your app, take the following steps.

    * Register an instance of the audio frame observer before joining a channel.
    * Set the format of audio frames captured by each callback.
    * Implement callbacks in the frame observers to process raw audio data.
    * Unregister the frame observers before you leave a channel.

    The following figure shows the basic processing of raw audio data:

    **Process raw audio**

    ![Raw Audio Processing](https://assets-docs.agora.io/images/video-sdk/process-raw-audio.svg)

    ## Prerequisites [#prerequisites-4]

    Ensure that you have implemented the [SDK quickstart](../index.mdx) in your project.

    ## Implement raw audio processing [#implement-raw-audio-processing-4]

    Follow these steps to implement raw audio data processing functionality in your app:

    1. Before joining a channel, create an `IAudioFrameObserver` instance and call the `registerAudioFrameObserver` method to register the audio frame observer.
    2. Call `setRecordingAudioFrameParameters`, `setPlaybackAudioFrameParameters`, and `setMixedAudioFrameParameters` methods to configure the audio frame format.
    3. Implement `onRecordAudioFrame`, `onPlaybackAudioFrame`, `onPlaybackAudioFrameBeforeMixing`, and `onMixedAudioFrame` callbacks. These callbacks receive and process audio frames. If the return value of these callbacks is `false`, it indicates that the processing of the audio frames is invalid.

    Refer to the following sample code to implement this logic:

    ```cpp
    BOOL CAgoraOriginalAudioDlg::RegisterAudioFrameObserver(
      BOOL bEnable, IAudioFrameObserver *audioFrameObserver) {

      agora::util::AutoPtr<agora::media::IMediaEngine> mediaEngine;
      // Query AGORA_IID_MEDIA_ENGINE interface
      mediaEngine.queryInterface(m_rtcEngine, agora::rtc::AGORA_IID_MEDIA_ENGINE);
      int nRet = 0;

      if (mediaEngine.get() == NULL) return FALSE;

      if (bEnable) {
        // Register the audio observer
        nRet = mediaEngine->registerAudioFrameObserver(audioFrameObserver);
      } else {
        // Unregister the audio observer
        nRet = mediaEngine->registerAudioFrameObserver(NULL);
      }

      return nRet == 0 ? TRUE : FALSE;
    }

    // Implement the onRecordAudioFrame callback
    bool COriginalAudioProcFrameObserver::onRecordAudioFrame(const char* channelId,
                                 AudioFrame& audioFrame) {
      SIZE_T nSize = audioFrame.channels * audioFrame.samplesPerChannel * 2;
      unsigned int readByte = 0;
      int timestamp = GetTickCount();
      short *pBuffer = (short *)audioFrame.buffer;

      for (SIZE_T i = 0; i < nSize / 2; i++) {
        if (pBuffer[i] * 2 > 32767) {
          pBuffer[i] = 32767;
        } else if (pBuffer[i] * 2 < -32768) {
          pBuffer[i] = -32768;
        } else {
          pBuffer[i] *= 2;
        }
      }

    #ifdef _DEBUG
      CString strInfo;
      strInfo.Format(_T("audio Frame buffer size:%d, timestamp:%d \n"), nSize, timestamp);
      OutputDebugString(strInfo);
      audioFrame.renderTimeMs = timestamp;
    #endif

      return true;
    }

    // Implement the onPlaybackAudioFrame callback
    bool COriginalAudioProcFrameObserver::onPlaybackAudioFrame(
      const char* channelId, AudioFrame& audioFrame) {
      return true;
    }

    // Implement the onMixedAudioFrame callback
    bool COriginalAudioProcFrameObserver::onMixedAudioFrame(const char* channelId, AudioFrame& audioFrame) {
      return true;
    }

    // Implement the onPlaybackAudioFrameBeforeMixing callback
    bool COriginalAudioProcFrameObserver::onPlaybackAudioFrameBeforeMixing(const char* channelId, rtc::uid_t uid, AudioFrame& audioFrame) {
      return true;
    }

    // Configure the audio frames captured by each callback
    m_rtcEngine->setRecordingAudioFrameParameters(44100, 2, RAW_AUDIO_FRAME_OP_MODE_READ_WRITE, 1024);
    m_rtcEngine->setPlaybackAudioFrameParameters(44100, 2, RAW_AUDIO_FRAME_OP_MODE_READ_WRITE, 1024);
    m_rtcEngine->setPlaybackAudioFrameBeforeMixingParameters(44100, 2);
    m_rtcEngine->setMixedAudioFrameParameters(44100, 2, 1024);
    ```

    <CalloutContainer type="warning">
      <CalloutDescription>
        Voice SDK uses a synchronous callback mechanism for processing raw audio data. When you save or rewrite data using the callbacks, consider the following best practices:

        * To ensure continuity of the audio stream, do not block the SDK thread by processing data directly in the callback function. Instead, make a deep copy of the received audio data and transfer the copied data to another thread for processing.
        * If you choose to process the audio data synchronously within the callback function, you must strictly control the processing time. For example, if the callback function is triggered every 10 milliseconds, then the processing time within the callback must be less than 10 milliseconds to prevent delays or interruptions in the audio stream.
      </CalloutDescription>
    </CalloutContainer>

    ## Reference [#reference-4]

    This section contains content that completes the information on this page, or points you to documentation that explains other aspects to this product.

    * [Audio module](../../core-concepts.mdx)

    ### Sample project [#sample-project-3]

    Agora provides an open-source example project [ProcessRawAudioData](https://gitee.com/agoraio-community/API-Examples/tree/main/windows/APIExample/APIExample/Advanced/OriginalAudio) for your reference. Download or view the project for a more detailed example.

    ### API reference [#api-reference-4]

    * [`registerAudioFrameObserver`](https://api-ref.agora.io/en/video-sdk/cpp/4.x/API/class_imediaengine.html#api_imediaengine_registeraudioframeobserver)
    * [`setRecordingAudioFrameParameters`](https://api-ref.agora.io/en/video-sdk/cpp/4.x/API/class_irtcengine.html#api_irtcengine_setrecordingaudioframeparameters)
    * [`setPlaybackAudioFrameParameters`](https://api-ref.agora.io/en/video-sdk/cpp/4.x/API/class_irtcengine.html#api_irtcengine_setplaybackaudioframeparameters)
    * [`setMixedAudioFrameParameters`](https://api-ref.agora.io/en/video-sdk/cpp/4.x/API/class_irtcengine.html#api_irtcengine_setmixedaudioframeparameters)
    * [`setPlaybackAudioFrameBeforeMixingParameters`](https://api-ref.agora.io/en/video-sdk/cpp/4.x/API/class_irtcengine.html#api_irtcengine_setplaybackaudioframebeforemixingparameters)
    * [`onRecordAudioFrame`](https://api-ref.agora.io/en/video-sdk/cpp/4.x/API/class_iaudioframeobserverbase.html#callback_iaudioframeobserverbase_onrecordaudioframe)
    * [`onPlaybackAudioFrame`](https://api-ref.agora.io/en/video-sdk/cpp/4.x/API/class_iaudioframeobserverbase.html#callback_iaudioframeobserverbase_onplaybackaudioframe)
    * [`onPlaybackAudioFrameBeforeMixing`](https://api-ref.agora.io/en/video-sdk/cpp/4.x/API/callback_iaudioframeobserver_onplaybackaudioframebeforemixing.html)
    * [`onMixedAudioFrame`](https://api-ref.agora.io/en/video-sdk/cpp/4.x/API/class_iaudioframeobserverbase.html#callback_iaudioframeobserverbase_onmixedaudioframe)