mod_audio_stream

mod_audio_stream is a production-grade WebSocket audio streaming module for FreeSWITCH. It enables real-time audio streaming between FreeSWITCH and external systems using an RFC-6455 compliant WebSocket client (libwsc).

Key Capabilities:

• Uni-directional streaming: Suitable for ASR, transcription, and analytics.
• Bi-directional streaming (v1.1.0+): Provides full-duplex audio (caller to WebSocket and vice-versa) with automatic resampling and dynamic playback control.
• Audio Formats: Supports base64-encoded and raw binary audio. Returned audio can be in PCMU, PCMA, L16/RAW, or Opus formats.
• Unified Recording: Supports uuid_record to capture both caller audio and returned WebSocket audio in a single recording.

Licensing Note:

• The pre-built release is free for use (including commercial) but is limited to 10 concurrent streaming channels.
• For unlimited channels or source-code licensing, contact amsoftswitch@gmail.com.

The module supports receiving audio from the WebSocket server to be played back to the user. The server must return a JSON object in the following format:

{
  "type": "streamAudio",
  "data": {
    "audioDataType": "raw",
    "sampleRate": 8000,
    "audioData": "base64 encoded audio"
  }
}

When the mod_audio_stream::play event is generated, the module converts the base64 data into a temporary file. The event body will contain the audioDataType, sampleRate, and a file key pointing to the path of the temporary file:

{
  "audioDataType": "raw",
  "sampleRate": 8000,
  "file": "/tmp/some_temp_file"
}

Note: All files generated for playback reside in a temporary directory and are deleted when the session is closed. The audioDataType can be raw, wav, mp3, or ogg.

{
  "type": "streamAudio",
  "data": {
    "audioDataType": "raw",
    "sampleRate": 8000,
    "audioData": "SGVsbG8gd29ybGQ=
  }
}

Use the following channel variables to fine-tune WebSocket connections, audio buffering, and logging behavior:

Additional Details

Extra Headers

STREAM_EXTRA_HEADERS must be a JSON object where keys are header names and values are strings:

{
    "Header1": "Value1",
    "Header2": "Value2"
}

TLS (WSS) Configuration

• Use STREAM_TLS_CA_FILE to specify a CA certificate or bundle.
• Use STREAM_TLS_CERT_FILE and STREAM_TLS_KEY_FILE for client-side certificate authentication.
• Use STREAM_TLS_DISABLE_HOSTNAME_VALIDATION to bypass hostname verification.

The module exposes several FreeSWITCH commands to control the audio stream via the channel UUID.

Start Audio Stream

uuid_audio_stream <uuid> start <wss-url> <mix-type> <sampling-rate> <metadata> Attaches a media bug and streams audio in L16 format to the specified WebSocket URL.

• <uuid>: FreeSWITCH channel unique ID.
• <wss-url>: The WebSocket URL (ws:// or wss://).
• <mix-type>:
  • mono: Single channel with caller's audio.
  • mixed: Single channel with both caller and callee audio.
  • stereo: Two channels (caller in one, callee in the other).
• <sampling-rate>:
  • 8k: 8000 Hz.
  • 16k: 16000 Hz (will be resampled if necessary).
• <metadata>: (Optional) UTF-8 text sent before audio starts.

Send Text

uuid_audio_stream <uuid> send_text <metadata> Sends a UTF-8 text message to the WebSocket server.

Stop Stream

uuid_audio_stream <uuid> stop <metadata> Stops the audio stream and closes the connection. If <metadata> is provided, it is sent before closing.

Pause/Resume Stream

• uuid_audio_stream <uuid> pause: Pauses the audio stream.
• uuid_audio_stream <uuid> resume: Resumes the audio stream.

uuid_audio_stream <uuid> start wss://example.com/stream mixed 16k {"user_id": "123"}

The module generates several FreeSWITCH events based on the WebSocket connection state and server responses.

Connection Events

• mod_audio_stream::connect: Triggered when successfully connected. Body: {"status": "connected"}.
• mod_audio_stream::disconnect: Triggered when disconnected. Body includes a status and a message object containing a code (int) and reason (string).
• mod_audio_stream::error: Triggered on connection errors. Body includes a status, message.code (int), and message.error (string).

Data Events

• mod_audio_stream::json: Triggered when a message is received from the WebSocket endpoint. The body contains the raw server response.
• mod_audio_stream::play: Triggered when the WebSocket server returns audio to be played by the user.

Error Codes