LiveKit Swift SDK

repository·main·Indexed 19 days ago

https://github.com/livekit/client-sdk-swift

A SDK for adding real-time video, audio, and data features to iOS and macOS applications. It supports building multi-modal AI, live streaming, and video calling apps by connecting to LiveKit Cloud or self-hosted servers. Key features include Room management, VideoView rendering, AVAudioSession configuration, CallKit integration, and support for Swift 6 strict concurrency.

Tokens
8.5K
Snippets
22
Records
31
Agent score
65%

What's inside livekit-client-sdk-swift

  1. Handle Thread Safety and Concurrency

    main

    The LiveKit Swift SDK follows these threading rules:

    • UI Components: All operations on VideoView (reading/writing properties, etc.) must be performed on the main thread.
    • Core Classes: Most other core classes can be accessed from any thread.
    • Delegates: SDK delegates are called on an internal SDK thread. You must ensure any UI updates triggered by delegates are dispatched to the main thread using @MainActor or DispatchQueue.main.async.
    • Swift 6: The SDK is compiled with Swift 6.1 and full support for strict concurrency. Apps using Swift 6 language mode can access LiveKit classes without needing @preconcurrency or @unchecked Sendable.
  2. Configure Audio Engine Observers

    main

    The SDK manages the internal audio engine through an AudioEngineObserver chain. You can customize this chain to manage AVAudioSession manually or provide custom lifecycle hooks.

    • iOS, visionOS, tvOS default: [AudioManager.shared.audioSession, AudioManager.shared.mixer]
    • macOS default: [AudioManager.shared.mixer]

    If you want to manage AVAudioSession yourself on iOS/tvOS but keep the default mixer setup, keep the default observers and set isAutomaticConfigurationEnabled = false.

    Warning: Configure the chain once early in app startup. Avoid changing it while the engine is in use. Setting an empty array [] disables all observers, including session handling and mixer setup.

    // Default chain for iOS/visionOS/tvOS
    AudioManager.shared.set(engineObservers: [AudioManager.shared.audioSession, AudioManager.shared.mixer])
    
    // Default chain for macOS
    AudioManager.shared.set(engineObservers: [AudioManager.shared.mixer])
  3. Choose between In-app and Broadcast screen capture modes

    main

    LiveKit supports two modes for screen sharing on iOS via ReplayKit:

    1. In-app Capture (default): Shares content only within your app. It requires no extra configuration and prompts the user for permission once per app execution. Note: Application audio is not supported in this mode.
    2. Broadcast Capture: Shares system-wide content, allowing users to switch to other apps while sharing. This requires a Broadcast Upload Extension and presents a "Screen Broadcast" dialog to the user each time sharing is requested.
  4. Manage Memory for SDK Objects

    main

    When storing references to objects managed by the SDK (such as Participant or TrackPublication), use weak references (weak var).

    These objects become invalid when the Room disconnects and are released by the SDK. Holding strong references to them will prevent the Room and other internal objects from being properly deallocated.

  5. Install the LiveKit Swift SDK

    main

    LiveKit for Swift can be installed via Swift Package Manager (SPM) or CocoaPods. Swift Package Manager is the recommended method.

    Swift Package Manager (SPM)

    Via Package.swift Add the LiveKit dependency to your Package.swift file:

    let package = Package(
      ...
      dependencies: [
        .package(name: "LiveKit", url: "https://github.com/livekit/client-sdk-swift.git", .upToNextMajor("2.16.0")),
      ],
      targets: [
        .target(
          name: "MyApp",
          dependencies: ["LiveKit"]
        )
      ]
    )

    Via Xcode UI

    1. Go to Project Settings -> Swift Packages.
    2. Add a new package using the URL: https://github.com/livekit/client-sdk-swift.

    Pre-built XCFramework

    For faster integration and CI builds, use the pre-compiled binary distribution. This bundles LiveKit.xcframework and its dependencies (LiveKitWebRTC, RustLiveKitUniFFI).

    URL: https://github.com/livekit/client-sdk-swift-xcframework

    CocoaPods

    Note: CocoaPods support is deprecated and will become read-only in 2027. It is strongly recommended to migrate to Swift Package Manager.

  6. Use Always-Prepared Recording Mode

    main

    To minimize latency when publishing the microphone, you can pre-warm the audio engine and keep the mic input prepared in a muted state. This makes publishing the mic almost immediate.

    Trade-offs:

    • Pros: Faster mic publishing; persists across Room lifecycles.
    • Cons: Longer initial app startup; requires mic permission (system prompt may appear); the audio engine stays running (muted) even after disconnect.

    Note: If isAutomaticConfigurationEnabled is true, the SDK configures the session category to .playAndRecord.

    // Enable pre-warmed engine
    Task.detached {
        try? await AudioManager.shared.setRecordingAlwaysPreparedMode(true)
    }
    
    // Disable when no longer needed
    try await AudioManager.shared.setRecordingAlwaysPreparedMode(false)
  7. Run unit tests on a device using LKTestHost

    main

    To execute unit tests on a physical iOS/macOS device using the LKTestHost application, follow these steps:

    1. Open the Xcode Project.
    2. Select your physical device from the target selector.
    3. Click the "Build and then test current scheme" icon (or use the test command in Xcode).

    Note: When running on a physical device, ensure you allow local network access when prompted by the OS to allow the app to communicate with your development server.

  8. Integrate LiveKit with CallKit

    main

    To integrate with CallKit, you must coordinate the timing between AVAudioSession and the SDK's audio engine to prevent the engine from starting outside of CallKit's active window.

    1. Initialization: As early as possible (before connecting to a Room), disable automatic configuration and set engine availability to .none:
      AudioManager.shared.audioSession.isAutomaticConfigurationEnabled = false
      try AudioManager.shared.setEngineAvailability(.none)
    2. Coordination: In your CXProviderDelegate, manage the engine availability based on the session state:
      • In provider(_:didActivate:): Configure the session category and set engine availability to .default.
      • In provider(_:didDeactivate:): Set engine availability to .none.
    // 1. Early setup
    AudioManager.shared.audioSession.isAutomaticConfigurationEnabled = false
    try AudioManager.shared.setEngineAvailability(.none)
    
    // 2. In CXProviderDelegate
    func provider(_: CXProvider, didActivate session: AVAudioSession) {
      do {
        try session.setCategory(.playAndRecord, mode: .voiceChat, options: [.mixWithOthers])
        try AudioManager.shared.setEngineAvailability(.default)
      } catch {
        // Handle error
      }
    }
    
    func provider(_: CXProvider, didDeactivate _: AVAudioSession) {
      do {
        try AudioManager.shared.setEngineAvailability(.none)
      } catch {
        // Handle error
      }
    }
  9. Configure the LKTestHost environment for on-device testing

    main

    LKTestHost is a minimal application used to run unit tests on a physical device. To connect to a LiveKit server, you must configure the following environment variables in the Xcode scheme arguments tab for the "LKTestHost" scheme:

    KeyDefaultDescription
    LIVEKIT_TESTING_URLws://localhost:7880The WebSocket URL of your LiveKit server
    LIVEKIT_TESTING_API_KEYdevkeyThe API key for the server
    LIVEKIT_TESTING_API_SECRETsecretThe API secret for the server

    If you are running a development server on your Mac and testing on a physical device on the same local network, set LIVEKIT_TESTING_URL to ws://<your-mac-lan-ip>:7880.

    # Example: Running a dev server accessible to the local network
    livekit-server --dev --bind 0.0.0.0
  10. Capture audio buffers in Manual Mode (without microphone access)

    main

    For apps that need to provide audio to a LiveKit room without requesting permission to use the physical microphone, you can use Manual Rendering Mode.

    Warning: In manual mode, the audio engine does not access any audio devices. Remote audio will not be played automatically; you are responsible for handling audio playback yourself.

    To set up manual mode:

    1. Enable manual rendering mode via AudioManager.shared.setManualRenderingMode(true).
    2. Enable the microphone track on the local participant (this enables the track in the room without triggering physical microphone access).
    3. Continuously provide audio buffers using AudioManager.shared.mixer.capture(appAudio:).
    // 1. Enable manual rendering mode
    try AudioManager.shared.setManualRenderingMode(true)
    
    // 2. Enable the microphone track (does not access physical hardware in manual mode)
    try await room.localParticipant.setMicrophone(enabled: true)
    
    // 3. Provide audio buffers continuously
    AudioManager.shared.mixer.capture(appAudio: yourAudioBuffer)
  11. Disable automatic AVAudioSession configuration

    main

    By default, the SDK automatically configures the AVAudioSession. If you are using frameworks like CallKit or have your own custom AVAudioSession management, you can prevent the SDK from interfering by disabling automatic configuration.

    Note: If you disable automatic configuration, you are responsible for setting the session category to .playAndRecord before unmuting or publishing the microphone.

    AudioManager.shared.audioSession.isAutomaticConfigurationEnabled = false
  12. Configure AudioSession Management

    main

    By default, LiveKit automatically manages the AVAudioSession. It typically uses the .playback category and switches to .playAndRecord when a local track is published.

    To take manual control of the AVAudioSession:

    1. Disable automatic configuration: AudioManager.shared.audioSession.isAutomaticConfigurationEnabled = false
    2. Ensure the session is configured with category .playAndRecord and mode .voiceChat or .videoChat before publishing the microphone.

    Optimization Tips:

    • Reduce Latency: To reduce microphone publishing latency, pre-warm the audio engine using: try await AudioManager.shared.setRecordingAlwaysPreparedMode(true).
    • Custom Observers: You can provide a custom AudioEngineObserver chain via AudioManager.shared.set(engineObservers:) to monitor the audio engine lifecycle.
    // Disable automatic configuration to manage AVAudioSession manually
    AudioManager.shared.audioSession.isAutomaticConfigurationEnabled = false
    
    // Pre-warm the engine to reduce mic publish latency
    try await AudioManager.shared.setRecordingAlwaysPreparedMode(true)