songbird

The Driver subdivides voice connection handling into several specialized tasks to balance performance and responsiveness:

• Core: Manages connection/reconnection and directs commands to network tasks.
• Mixer (Synchronous): Combines audio sources, performs Opus encoding, and encrypts packets every 20ms. It handles track commands and transmits voice packets.
• Thread Pool (Synchronous): A dynamic pool for I/O tasks, such as Seek operations or creating lazy tracks via Compose.
• Disposer (Synchronous): Used by the mixer thread to drop audio sources that might have long or blocking Drop implementations.
• Events: Manages event handlers and tracks event timing.
• Websocket (Network Task): Sends speaking status/keepalives to Discord and receives connection events.
• UDP Rx (Optional Network Task): Decrypts and decodes incoming voice packets and statistics.

All tasks communicate via interconnecting channels.

To optimize memory and thread usage, Songbird uses a specialized scheduler for Mixer tasks:

• Parking: Mixer tasks without live tracks are "parked" in a single async task to save resources. They are moved to a live thread only when a Track is added.
• Live Threads: When a mixer is active, it runs on a live thread. Each thread can run multiple mixers (defaulting to 16 per thread, but user-configurable) in a 20ms tick.
• Execution Budget: Audio threads aim to complete mixing, encoding, and encryption within an 18ms budget. If work exceeds this, the scheduler offloads the highest-cost task to stay within the 20ms deadline.
• Mixing Order: To minimize latency variance, the mixer follows a strict order: handle idle/live messages $\rightarrow$ handle driver/mixer messages $\rightarrow$ cleanup $\rightarrow$ mix/encode/encrypt $\rightarrow$ check packet blocks $\rightarrow$ sleep $\rightarrow$ send packets $\rightarrow$ handle per-track messages.

Songbird manages audio through two main abstractions:

Inputs

Inputs are audio sources that support lazy initialization:

• Lazy Inputs: Trait objects that allow instructions to store a way to create an audio source cheaply. They are initialized only when needed.
• Live Inputs: Objects implementing MediaSource: Read + Seek.

Internally, the mixer uses floating-point audio to prevent clipping. Songbird uses Symphonia to decode various formats. If a source is already in Opus format and is the only source playing, Songbird can bypass mixing and re-encoding to save CPU.

Tracks

Tracks manage the state of a playing source, including position, play state, and modifiers (like volume).

• Modification: While tracks are defined in user code, once passed to the driver, all changes must be requested via a TrackHandle. This ensures the audio thread never locks or blocks while the user modifies track properties.

Songbird requires several system-level dependencies to function correctly.

1. Opus (Required)

Songbird cannot work without the Opus audio codec.

• Linux (Ubuntu/Debian): sudo apt install libopus-dev
• Linux (Arch): sudo pacman -S opus
• macOS/Linux (via pkgconf): It will attempt to use installed libopus binaries.
• Windows: You must have cmake installed to build opus from source.
• Fallback: If binaries are not found, Songbird will attempt to build from source using a C compiler and GNU autotools (build-essential, autoconf, automake, libtool, m4 on Ubuntu; base-devel on Arch).

2. yt-dlp / youtube-dl (Optional for users, Required for dev-dependencies)

Required if you want Songbird to download audio/video from the internet (e.g., YouTube) and convert it to Opus.

• via pip: pip install youtube_dl
• Ubuntu: sudo apt install youtube-dl
• Arch: sudo pacman -S youtube-dl

To use Songbird, add it to your Cargo.toml. By default, Songbird only supports Opus via the DCA file format. If you need support for other audio formats (like MP3, AAC, FLAC, etc.), you must also add symphonia as a dependency in your project and enable the required features.

Default Setup (DCA/Opus only)

[dependencies.songbird]
version = "0.5"
features = ["builtin-queue"]

Extended Codec Setup (MP3, AAC, etc.)

To enable additional codecs, include symphonia with the desired features:

[dependencies.songbird]
version = "0.5"
features = ["builtin-queue"]

[dependencies.symphonia]
version = "0.5"
features = ["aac", "mp3", "isomp4", "alac"]

# Including songbird alone gives you support for Opus via the DCA file format.
[dependencies.songbird]
version = "0.5"
features = ["builtin-queue"]

# To get additional codecs, you *must* add Symphonia yourself.
[dependencies.symphonia]
version = "0.5"
features = ["aac", "mp3", "isomp4", "alac"]

When receiving audio from Discord, you can specify how RTP packets are handled using the DecodeMode enum. This determines the CPU overhead and the state of the audio data passed to your event handlers.

• DecodeMode::Pass: No changes are applied to the packets. This involves no CPU work but provides raw, encrypted packets.
• DecodeMode::Decrypt: Decrypts the body of each received packet. This has a small per-packet CPU cost.
• DecodeMode::Decode(DecodeConfig): Decrypts and decodes each packet, accounting for packet loss. This has the highest per-packet CPU cost but provides usable audio samples.

You can check the capabilities of a mode using should_decode() and should_decrypt().

use songbird::driver::decode_mode::{DecodeMode, DecodeConfig, Channels, SampleRate};

// Example: Using Decode mode with specific audio settings
let config = DecodeConfig::new(Channels::Stereo, SampleRate::Hz48000);
let mode = DecodeMode::Decode(config);

if mode.should_decode() {
    // Handle decoded audio
}