LiveKit Flutter SDK

repository·main·Indexed 19 days ago

https://github.com/livekit/client-sdk-flutter

The LiveKit Flutter SDK (livekit_client ^2.10.0) enables the integration of real-time video, audio, and data features into Flutter applications. It supports connecting to LiveKit Cloud or self-hosted servers for video calling, live streaming, and multi-modal AI. Key features include end-to-end encryption (E2EE), screen sharing across platforms, certificate pinning for native platforms, and tools for managing audio sessions and video track rendering.

Tokens
18.1K
Snippets
58
Records
73
Agent score
64%

What's inside livekit-client-sdk-flutter

  1. React to Room and Participant changes

    main

    The SDK provides two ways to build reactive UIs:

    1. ChangeNotifier: Use addListener on Room or Participant for generic updates that might impact rendering.
    2. EventsListener<Event>: Use createListener() to listen for specific events (e.g., ParticipantConnectedEvent, RoomDisconnectedEvent). Always call dispose() on the listener to prevent leaks.
    // Using EventsListener for specific events
    late final EventsListener<RoomEvent> _listener = widget.room.createListener();
    
    @override
    void initState() {
      super.initState();
      _listener
        ..on<RoomDisconnectedEvent>((_) {
          // handle disconnect
        })
        ..on<ParticipantConnectedEvent>((e) {
          print("participant joined: ${e.participant.identity}");
        });
    }
    
    @override
    void dispose() {
      _listener.dispose();
      super.dispose();
    }
  2. Configure Certificate Pinning

    main

    Certificate pinning is available on native platforms via RoomOptions.networkOptions. It applies to SDK-owned WSS signaling and internal HTTPS requests, but not to WebRTC media, TURN, or application-owned token endpoints.

    Key Rules:

    • Web Support: Pinning is not supported on Flutter web. Configuring it on web will cause Room.connect to throw an UnsupportedError.
    • Host Matching: Supports exact hosts, single-label wildcards (*.livekit.cloud), and multi-label wildcards (**.livekit.cloud). Use **.livekit.cloud to ensure coverage for LiveKit Cloud regional failover hosts.
    • SPKI Pins: Use SPKI SHA-256 pins in primaryPins and backupPins. The SDK matches these against the leaf certificate's public key only. Do not pin intermediate or root CA keys, as Dart does not expose the full chain.
    • Leaf Certificates: Use pinnedLeafCertificates to require an exact peer leaf certificate. If using private PKI, you must also configure trustedCertificates to anchor the validation.
    // Example: SPKI Pinning
    final roomOptions = RoomOptions(
      networkOptions: NetworkOptions(
        certificatePinning: CertificatePinningOptions(
          rules: [
            CertificatePinningRule(
              hosts: ['**.livekit.cloud'],
              primaryPins: ['sha256/current-public-key-pin'],
              backupPins: [
                'sha256/next-public-key-pin-1',
                'sha256/next-public-key-pin-2',
              ],
            ),
          ],
        ),
      ),
    );
    
    final room = Room(roomOptions: roomOptions);
    await room.connect(url, token);
    
    // Example: Exact Leaf Certificate and Custom Trust Store
    final certificate = await CertificateBytes.fromAsset(
      'assets/livekit_leaf_cert.pem',
    );
    
    final roomOptions = RoomOptions(
      networkOptions: NetworkOptions(
        certificatePinning: CertificatePinningOptions(
          rules: [
            CertificatePinningRule(
              hosts: ['my-project.livekit.cloud'],
              pinnedLeafCertificates: [certificate],
              trustedCertificates: [certificate],
            ),
          ],
        ),
      ),
    );
  3. Manage audio sessions with AudioManager

    main

    LiveKit uses the AudioManager singleton to manage platform audio sessions (iOS/Android) and audio engine state.

    Automatic Mode (Default)

    By default, LiveKit manages the session automatically using a communication intent. This is ideal for standard calls and requires no setup. On iOS, this uses playAndRecord when the microphone is active and playback for listen-only playout. On Android, it uses communication mode with voice call routing and audio focus.

    Manual Mode

    To take full control of the session (e.g., for a playback-only app or custom platform configurations), switch to manual mode. In manual mode, LiveKit stops managing the session based on room or engine lifecycle, and your app becomes responsible for the session.

    Key Capabilities

    • Session Control: Switch between automatic and manual modes.
    • Speaker Routing: Control whether audio prefers the speaker or a headset.
    • State Observation: Monitor the native audio engine state (playout and recording status).
    • Audio Processing: Manage signal processing like echo cancellation and noise suppression.
    import 'package:livekit_client/livekit_client.dart';
    
    // Access the singleton
    final audioManager = AudioManager.instance;
    
    // Switch to manual mode
    await audioManager.setAudioSessionManagementMode(AudioSessionManagementMode.manual);
    
    // Switch back to automatic mode
    await audioManager.setAudioSessionManagementMode(AudioSessionManagementMode.automatic);
  4. Migrate from legacy Hardware audio APIs to AudioManager

    main

    The legacy Hardware audio members are deprecated and have been moved to AudioManager. Use the following mapping to update your code:

    Old APINew API
    Hardware.instance.setSpeakerphoneOn(true)AudioManager.instance.setSpeakerOutputPreferred(true)
    room.setSpeakerOn(true)AudioManager.instance.setSpeakerOutputPreferred(true)
    Hardware.instance.speakerOnAudioManager.instance.isSpeakerOutputPreferred
    Hardware.instance.preferSpeakerOutputAudioManager.instance.isSpeakerOutputPreferred
    Hardware.instance.forceSpeakerOutputAudioManager.instance.isSpeakerOutputForced
    Hardware.instance.setAutomaticConfigurationEnabled(enable: false)AudioManager.instance.setAudioSessionManagementMode(AudioSessionManagementMode.manual)

    Note on configuration: The onConfigureNativeAudio hook is removed. Instead of using a custom configuration function, use explicit options via setAudioSessionOptions, which triggers manual mode.

    // Before: assigning onConfigureNativeAudio with a custom function.
    // After:
    await AudioManager.instance.setAudioSessionOptions(
      AudioSessionOptions.communication(
        apple: const AppleAudioSessionConfiguration(
          category: AppleAudioCategory.playAndRecord,
          mode: AppleAudioMode.videoChat,
        ),
      ),
    );
  5. Customize iOS launch screen assets

    main

    To change the image displayed during the app's launch on iOS, you can replace the existing image files within the example/ios/Runner/Assets.xcassets/LaunchImage.imageset/ directory with your own assets.

    Alternatively, you can manage these assets using Xcode:

    1. Open your Flutter project's iOS workspace using open ios/Runner.xcworkspace.
    2. In the Xcode Project Navigator, navigate to Runner/Assets.xcassets.
    3. Drag and drop your desired images into the asset catalog to replace the launch images.
    open ios/Runner.xcworkspace
  6. Generate an SPKI pin using OpenSSL

    main

    To generate a SHA-256 SPKI pin for your primaryPins or backupPins, run the following command and prefix the output with sha256/.

    openssl s_client -connect your-host:443 -servername your-host </dev/null 2>/dev/null \
      | openssl x509 -pubkey -noout \
      | openssl pkey -pubin -outform der \
      | openssl dgst -sha256 -binary \
      | openssl base64
  7. Configure per-platform audio session overrides

    main

    When preset constructors like AudioSessionOptions.communication() are insufficient, you can pin exact platform values using AudioManager.instance.setAudioSessionOptions.

    Important: Supplying options via setAudioSessionOptions switches the SDK to manual mode.

    • Apple: The AppleAudioSessionConfiguration is applied verbatim. You must specify the category, categoryOptions, and mode.
    • Android: The AndroidAudioSessionConfiguration is applied. Any field left null is omitted, allowing the native manager to retain its current value for that field.
    await AudioManager.instance.setAudioSessionOptions(
      AudioSessionOptions.communication(
        apple: const AppleAudioSessionConfiguration(
          category: AppleAudioCategory.playAndRecord,
          categoryOptions: {
            AppleAudioCategoryOption.allowBluetooth,
            AppleAudioCategoryOption.mixWithOthers,
          },
          mode: AppleAudioMode.voiceChat,
        ),
        android: AndroidAudioSessionConfiguration.communication,
      ),
    );
  8. Compile E2EE Web Worker for Web Support

    main

    If you are targeting the Web platform and want to use End-to-End Encryption (E2EE), you must manually compile the E2EE web worker before deployment or running the web version.

    dart compile js web/e2ee.worker.dart -o example/web/e2ee.worker.dart.js -m
  9. Configure Android permissions

    main

    LiveKit depends on Flutter WebRTC, which requires several permissions to be declared in your AndroidManifest.xml. You must include permissions for camera, audio recording, network state, and Bluetooth.

    <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.your.package">
      <uses-feature android:name="android.hardware.camera" />
      <uses-feature android:name="android.hardware.camera.autofocus" />
      <uses-permission android:name="android.permission.CAMERA" />
      <uses-permission android:name="android.permission.RECORD_AUDIO" />
      <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
      <uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
      <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
      <uses-permission android:name="android.permission.BLUETOOTH" android:maxSdkVersion="30" />
      <uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
      <uses-permission android:name="android.permission.BLUETOOTH_ADMIN" android:maxSdkVersion="30" />
      ...
    </manifest>
  10. Configure iOS permissions and background modes

    main

    To use the camera and microphone on iOS, you must declare usage descriptions in your Info.plist.

    To support voice calls while the app is in the background, you must:

    1. Enable Background Modes in your Xcode project capabilities.
    2. Check Audio, AirPlay, and Picture in Picture.
    3. Add the audio string to the UIBackgroundModes array in your Info.plist.
    <dict>
      ...
      <key>NSCameraUsageDescription</key>
      <string>$(PRODUCT_NAME) uses your camera</string>
      <key>NSMicrophoneUsageDescription</key>
      <string>$(PRODUCT_NAME) uses your microphone</string>
      <key>UIBackgroundModes</key>
      <array>
        <string>audio</string>
      </array>
    </dict>
  11. Enable Screen Sharing

    main

    Screen sharing is supported across all platforms using room.localParticipant.setScreenShareEnabled(true). Platform-specific requirements apply:

    Android

    Requires a media projection foreground service. In AndroidManifest.xml, declare the service with mediaProjection type:

    <service
        android:name="de.julianassmann.flutter_background.IsolateHolderService"
        android:enabled="true"
        android:exported="false"
        android:foregroundServiceType="mediaProjection" />

    Crucial: You must call Helper.requestCapturePermission() from flutter_webrtc and verify it returns true before enabling screen share.

    iOS

    Requires a broadcast extension. Follow the flutter-webrtc iOS setup guide.

    Desktop (Windows/macOS)

    Use ScreenSelectDialog to allow users to pick a source, then create a track using LocalVideoTrack.createScreenShareTrack.

    // Desktop Screen Share Example
    try {
      final source = await showDialog<DesktopCapturerSource>(
        context: context,
        builder: (context) => ScreenSelectDialog(),
      );
      if (source == null) {
        print('cancelled screenshare');
        return;
      }
      print('DesktopCapturerSource: ${source.id}');
      var track = await LocalVideoTrack.createScreenShareTrack(
        ScreenShareCaptureOptions(
          sourceId: source.id,
          maxFrameRate: 15.0,
        ),
      );
      await room.localParticipant.publishVideoTrack(track);
    } catch (e) {
      print('could not publish screen sharing: $e');
    }
  12. Request Bluetooth permissions on Android

    main

    To ensure Bluetooth headsets work correctly on Android, use the permission_handler package to request bluetooth and bluetoothConnect permissions when the app launches.

    import 'package:permission_handler/permission_handler.dart';
    
    Future<void> _checkPermissions() async {
      var status = await Permission.bluetooth.request();
      if (status.isPermanentlyDenied) {
        print('Bluetooth Permission disabled');
      }
      status = await Permission.bluetoothConnect.request();
      if (status.isPermanentlyDenied) {
        print('Bluetooth Connect Permission disabled');
      }
    }
    
    void main() async {
      WidgetsFlutterBinding.ensureInitialized();
      await _checkPermissions();
      runApp(MyApp());
    }