TwovVoip GDExtension

The TwoVoIP extension relies on several submodules for audio processing and GDExtension bindings:

• godot-cpp: Provides the header files and class definitions for GDExtension. TwoVoIP 6.1 pins to godot-cpp version 10.0.0-rc1 (commit 58d1de720b8ffe9f8ffcdfe3a85148582cfd2e74), which is compatible with Godot 4.6-stable.
• opus: The Xiph opus library used for voice compression/decompression. It typically processes 20ms chunks of 48kHz stereo audio (960 float pairs) into 20-30 bytes of compressed data.
• speexdsp: The Xiph SpeexDSP library (pinned to commit 1b28a0f61bc31162979e1f26f3981fc3637095c8). TwoVoIP compiles the resampler and preprocessor sources directly into the extension using floating-point mode.
• rnnoise: The Xiph RNNoise library (pinned to commit 372f7b4b76cde4ca1ec4605353dd17898a99de38). TwoVoIP uses a CMake adapter in thirdparty/rnnoise to build it and automatically downloads and verifies the RNNoise model (revision 0b50c45) via SHA-256.

To build the TwoVoIP GDExtension from source on a NixOS system, you must use nix-shell to provide the necessary build dependencies (scons, cmake, ninja, and autoreconfHook), then run the scons build sequence. After building, you must copy the resulting shared objects (.so files) to the example directory to ensure the addon is usable in the example project.

Build Sequence:

1. Enter the build environment with nix-shell.
2. Apply portability and size patches using scons apply_patches.
3. Build the opus dependency using scons build_opus.
4. Build the rnnoise dependency using scons build_rnnoise.
5. Build the main library using scons.
6. Copy the compiled libraries to the example addon path.

nix-shell -p scons cmake ninja autoreconfHook
scons apply_patches  # apply the pinned portability and size patches
scons build_opus     # build opus using cmake
scons build_rnnoise  # build RNNoise using cmake
scons                # build this library
cp addons/twovoip/libs/*.so example/addons/twovoip/libs/

Use $TwoVoipMic.init_voip_mic() to set up the microphone input.

Parameters:

1. json_packets_as_binary (bool): If true, JSON headers/footers are stringified and emitted via the transmit_audio_packet signal instead of the transmit_audio_json_packet signal. This simplifies networking but prevents intercepting JSON metadata.
2. MicOn, PTT, Vox, Denoise (optional): UI buttons to control microphone state.
3. InputOption (OptionButton): An OptionButton populated with AudioServer.get_input_device_list().
4. voxshader.gdshader (ShaderMaterial): A material used to render an activity waveform.

Opus Configuration: Configure the encoder using set_opus_values(opussamplerate, opusframedurationms, channels, opusbitrate, opuscomplexity, opusoptimizeforvoice, denoiser, agc_mode):

• opussamplerate: [48000, 24000, 12000, 8000]
• opusframedurationms: [5, 10, 20, 40, 60]
• channels: 1 (mono) or 2 (stereo)
• opusbitrate: 500 to 64000
• opuscomplexity: 1 to 10
• opusoptimizeforvoice: boolean

Vox Settings:

• set_voxthreshhold(voxthreshhold): Sets the gating threshold (used by the shader).
• hangtime: Time the mic stays active after noise falls below threshold.
• leadtime: (Not implemented) Buffer time captured before threshold is reached.

# Example initialization concept
$TwoVoipMic.init_voip_mic(true, mic_on_btn, ptt_btn, vox_btn, denoise_btn, input_option_button, vox_shader_material)
$TwoVoipMic.set_opus_values(48000, 20, 1, 32000, 5, true, denoiser_mode, agc_mode)
$TwoVoipMic.set_voxthreshhold(0.5)

The provided demo project (radiomqtt.tscn) includes several panels to test VoIP functionality:

Microphone and Sound Output

• Mic Enabled: A green indicator shows the microphone is active.
• PTT (Push to Talk): Manually triggers recording and transmission.
• Vox (Voice Activated): Enables voice activation. The sensitivity is controlled by the size of the pink area on the blue sound visualizer.
• De-Noise: Toggles the RNNoise filter on or off.

LibOpus Compression and Resampling

You can adjust the following Opus parameters to test bandwidth vs. quality trade-offs:

• Frame duration: Settings between 2.5ms and 60ms.
• Bit rate: The target bandwidth.
• Sample rate
• Compression Complexity
• Stereo: Toggle for stereo recording.
• Optimized for voice: Toggle for voice-specific optimization.

Note: Changing these values or toggling De-Noise reprocesses the current microphone recording to allow immediate comparison.

Recording Playback

• Play: Decodes and plays back the most recent recorded sample.
• Stats: Displays the number of packets and bytes that would be transmitted for the current clip.

MQTT Transmission

This section uses the godot-mqtt GDExtension to push audio packets over a network via a broker.

• Connect: Connects to the MQTT broker. If running in HTML5, it automatically uses WebSockets.
• User Table: Shows connected users and their transmission status.
• Tone Test (T button): Replaces incoming audio with a pure 440Hz tone. This helps distinguish between network packet loss (gaps in audio) and poor source audio quality.

For manual control, use TwovoipOpusEncoder. This allows you to manage the audio pipeline directly.

Workflow:

1. Initialize: Call initialize() with sample rate, Opus rate, channels, denoiser mode, AGC mode, and an initial chunk size.
2. Configure Opus: Call create_opus_encoder() with desired bitrate, frame duration, and voice optimization.
3. Process Loop:
   • Get required input size via get_required_input_chunk_size().
   • Fetch frames from AudioServer.get_input_frames(required).
   • Call process_chunk(frames) to consume input. Note: You must retain any unconsumed frames (frames.size() - consumed) as the encoder does not buffer them.
   • Call encode_chunk() to get the Opus packet.

Important Notes:

• get_current_chunk(): Returns processed PCM as PackedVector2Array (stereo). Mono is duplicated into both channels.
• get_required_input_chunk_size(): Constant for the object's lifetime; represents the ceiling of the input/output sample ratio.
• get_speech_probability(): Returns a normalized 0–1 value (RNNoise) or percentage (Speex).

var encoder := TwovoipOpusEncoder.new()
var error := encoder.initialize(
    AudioServer.get_input_mix_rate(),
    48000,
    2,
    TwovoipOpusEncoder.DENOISER_DISABLED,
    TwovoipOpusEncoder.AGC_DISABLED,
    960,
)
assert(error == OK)
encoder.create_opus_encoder(12000, 5, true)

var required := encoder.get_required_input_chunk_size()
var frames := AudioServer.get_input_frames(required)
var consumed := encoder.process_chunk(frames)
if consumed >= 0:
    var peak := encoder.get_peak()
    var rms := encoder.get_rms()
    var conditioned_pcm := encoder.get_current_chunk()
    var packet := encoder.encode_chunk()