LiveKit Rust SDKs

The PlatformAudio FFI interface uses a reference-counting mechanism to manage the underlying Audio Device Module (ADM). Multiple FFI clients can create handles, but they all share the same underlying ADM instance.

Lifecycle Rules

• Creation: The first call to NewPlatformAudioRequest triggers the creation and initialization of the Platform ADM and enables ADM recording.
• Sharing: Subsequent calls to NewPlatformAudioRequest increment the internal reference count and reuse the existing ADM.
• Termination: The ADM is only terminated and disabled when the reference count reaches zero (i.e., the last handle is released via DisposeRequest).

Reference Counting Example

1. Client A calls NewPlatformAudioRequest() $\rightarrow$ handle_1 created (ref_count: 1).
2. Client B calls NewPlatformAudioRequest() $\rightarrow$ handle_2 created (ref_count: 2).
3. Client A calls DisposeRequest(handle_1) $\rightarrow$ (ref_count: 1).
4. Client B calls DisposeRequest(handle_2) $\rightarrow$ (ref_count: 0, ADM disabled).

Platform Audio Mode is used for dedicated VoIP applications that require direct access to the microphone and speakers, and need Acoustic Echo Cancellation (AEC). In this mode, the SDK manages the hardware via a platform-specific Audio Device Module (ADM).

Initialization Flow

1. PlatformAudio::new() is called.
2. The runtime acquires the platform ADM via AdmProxy::AcquirePlatformAdm(), which increments the reference count and creates the webrtc::AudioDeviceModule if it's the first time.
3. Recording is enabled via runtime.set_adm_recording_enabled(true).
4. Playout is enabled via runtime.set_adm_playout_enabled(true).

Outbound Audio (Microphone to Network)

The Platform ADM captures microphone PCM data. This data is routed through AudioState to AudioSendStream instances where external=false (device sources). The audio is then encoded and sent over the network.

Inbound Audio (Network to Speakers)

Decoded audio from the network flows through AudioReceiveStream into the AudioMixer. Because playout_enabled is true, the AdmProxy's NeedMorePlayData() delegates the audio to the platform_adm_, which plays the audio directly to the hardware speakers.

Key Characteristics

• Platform ADM: Created and managed by the SDK.
• AEC Support: Acoustic Echo Cancellation works because the playout signal is routed through the ADM, providing the necessary reference signal.
• iOS Behavior: AVAudioSession is configured for VoIP mode.

The AdmProxy uses a reference counting pattern to manage the lifecycle of the Platform ADM. This allows multiple clients (e.g., different FFI handles or multiple users in a single application) to share the same hardware audio resources without redundant initialization or premature termination.

• Single User: PlatformAudio::new() increments the ref count to 1 and creates the ADM. drop(audio) decrements it to 0 and terminates the ADM.
• Multiple Users: If audio1 is active and audio2 = PlatformAudio::new() is called, the ref count becomes 2. The ADM is reused. The ADM is only terminated when the last active handle is dropped.
• FFI Clients (Unity/Python): Multiple handles (e.g., from different Unity clients) increment the platform_adm_ref_count_. A separate process (like a Python agent) using a NativeAudioSource does not interact with the platform_adm_ and thus operates in Synthetic Mode, even if Unity clients have activated Platform Mode.

Reference Counting Examples

  SCENARIO 1: Single User
  ════════════════════════

    Time ──────────────────────────────────────────────────────────▶

    ┌──────────────────────┐                    ┌──────────────────────┐
    │ PlatformAudio::new() │                    │ drop(audio)          │
    │ ref_count: 0 → 1     │                    │ ref_count: 1 → 0     │
    │ CREATE Platform ADM  │                    │ TERMINATE Platform   │
    └──────────┬───────────┘                    └──────────┬───────────┘
               │                                           │
               ▼                                           ▼
    ═══════════╪═════════════════════════════════════════╪═══════════
    Synthetic  │         Platform Mode Active              │  Synthetic
               │                                           │

  SCENARIO 2: Multiple Users (Shared ADM)
  ═════════════════════════════════════════

    Time ──────────────────────────────────────────────────────────▶

    ┌───────────┐     ┌───────────┐     ┌───────────┐     ┌───────────┐
    │ audio1 =  │     │ audio2 =  │     │ drop(audio1│     │ drop(audio2│
    │ new()     │     │ new()     │     │           │     │           │
    │ ref: 0→1  │     │ ref: 1→2  │     │ ref: 2→1  │     │ ref: 1→0  │
    │ CREATE    │     │ (reuse)   │     │ (still    │     │ TERMINATE │
    │ ADM       │     │           │ │  active)  │     │ ADM       │
    └─────┬─────┘     └─────┬─────┘     └─────┬─────┘     └─────┬─────┘
          │                 │                 │                 │
          ▼                 ▼                 ▼                 ▼
    ════╪═════════════════╪═════════════════╪═════════════════╪══════
    Synth │    Platform Mode Active           │                 │ Synth
          │                                   │                 │ audio2 still
          │                                   │                 │ works!

  SCENARIO 3: FFI Clients (Unity/Python)
  ═════════════════════════════════════════

    ┌─────────────────┐       ┌─────────────────┐       ┌─────────────────┐
    │ Unity Client A  │       │ Unity Client B  │       │ Python Agent    │
    │                 │       │                 │                 │
    │ NewPlatformAudio│       │ NewPlatformAudio│       │ Uses Native     │
    │ Request         │       │ Request         │       │ AudioSource     │
    │ handle_1         │       │ handle_2         │       │ (no PlatformAdm)│
    └────────┬────────┘       └────────┬────────┘       └─────────────────┘
             │                         │                         
             ▼                         ▼                         
    ┌─────────────────────────────────────────────────────────────────┐
    │                        AdmProxy                                  │
    │                                                                  │
    │   platform_adm_ref_count_ = 2  (from Unity clients)              │
    │                                                                  │
    │   Both Unity clients share the same Platform ADM.                │
    │   Python agent uses synthetic mode (NativeAudioSource).          │
    │                                                                  │
    │   When both Unity clients call DisposeRequest:                   │
    │     handle_1 dispose → ref_count = 1                             │
    │     handle_2 dispose → ref_count = 0 → TERMINATE                 │
    └─────────────────────────────────────────────────────────────────┘