mediasoup-client Documentation

repository·v3·Indexed 20 days ago

https://github.com/versatica/mediasoup-client

A TypeScript client-side library for building WebRTC applications powered by the mediasoup SFU. It provides tools to manage device capabilities, create send and receive transports, and handle media production and consumption via the Device, Consumer, DataConsumer, and DataProducer classes.

Tokens
9.5K
Snippets
26
Records
43
Agent score
69%

What's inside mediasoup-client

  1. How Device and Transport work together

    v3

    The Device is the central entry point for mediasoup-client. It represents the client-side capabilities and manages the creation of transports.

    • Device.load({ routerRtpCapabilities }): This method initializes the device with the capabilities of the remote mediasoup router. This is a prerequisite for any media operations.
    • Device.createSendTransport(options): Creates a local representation of a server-side send transport. The options must include parameters like id, iceParameters, iceCandidates, dtlsParameters, and sctpParameters obtained from the server.
    • Transport Events: Transports are not fully functional until their event handlers are wired to your signaling server. The most critical events are:
      • connect: Triggered when the transport needs to exchange DTLS parameters to establish a secure connection.
      • produce: Triggered when a producer (like a video track) is being created and needs to send its RTP parameters to the server.
      • producedata: Triggered when a data producer (DataChannel) is being created.
  2. Use the Transport class to manage media and data

    v3

    The Transport class is the primary interface for managing WebRTC connections in mediasoup-client. It handles both media (RTP) and data (SCTP) flows. A transport can be either a sending transport (direction: 'send') or a receiving transport (direction: 'recv').

    Key capabilities include:

    • Media Production: Create Producer instances to send audio or video tracks.
    • Media Consumption: Create Consumer instances to receive remote audio or video tracks.
    • Data Communication: Create DataProducer and DataConsumer instances for SCTP-based data channels.
    • Connection Management: Monitor connection states, ICE gathering, and restart ICE connections.
    // Conceptual usage pattern
    const transport = new Transport({
      direction: 'send',
      id: 'my-transport-id',
      iceParameters: { ... },
      iceCandidates: [ ... ],
      dtlsParameters: { ... },
      // ... other required options
    });
    
    // Listen for connection events
    transport.on('connect', ({ dtlsParameters }, callback, error) => {
      // Handle connection
    });
  3. Use the Consumer class to manage remote media tracks

    v3

    The Consumer class represents a remote media track being received by the client. It provides control over the media stream (pausing/resuming) and allows you to monitor the lifecycle of the track through events.

    Key capabilities include:

    • Controlling playback: Use .pause() to disable the track and .resume() to re-enable it.
    • Monitoring state: Access properties like .id, .producerId, .kind, and .track.
    • Retrieving statistics: Use .getStats() to get an RTCStatsReport for the associated receiver.
    • Lifecycle management: Call .close() to manually terminate the consumer and clean up the track.
    // Example conceptual usage
    const consumer = new Consumer({
      id: 'consumer-id',
      localId: 'local-id',
      producerId: 'producer-id',
      track: mediaStreamTrack,
      rtpParameters: rtpParams
    });
    
    consumer.on('trackended', () => {
      console.log('Remote track ended');
    });
    
    consumer.pause();
    // ... later
    consumer.resume();
  4. Understand RtpCapabilities

    v3

    RTP capabilities define what a mediasoup router or an endpoint can receive at the media level. They consist of supported codecs and RTP header extensions.

    An RtpCapabilities object contains:

    • codecs: An array of RtpCodecCapability objects describing supported media and RTX codecs.
    • headerExtensions: An array of RtpHeaderExtension objects describing supported RTP header extensions.
  5. Understand RtpParameters

    v3

    The RtpParameters object describes a media stream. It differs slightly depending on whether it is for sending or receiving:

    RTP Send Parameters

    Used for media received by mediasoup from an endpoint (via a Producer).

    • Supports multiple encodings for simulcast. Each encoding must include an ssrc or an rid.
    • May include a mid (MID RTP extension value).

    RTP Receive Parameters

    Used for media sent by mediasoup to an endpoint (via a Consumer).

    • Contains a single entry in the encodings array, even if the producer uses simulcast. Spatial/temporal layer selection is handled via consumer.setPreferredLayers().
    • The mid value is unset.
    • ssrc values are randomly generated for all encodings, regardless of the original producer's parameters.
  6. Initialize a Device using Device.factory()

    v3

    The recommended way to create a Device instance is using the asynchronous Device.factory() method. This method automatically detects the browser/environment and selects the appropriate built-in RTC handler. If the environment is not supported, it throws an UnsupportedError.

    You can also provide a specific handlerName or a custom handlerFactory via DeviceOptions if you want to bypass automatic detection or use a custom implementation.

    import { Device } from 'mediasoup-client';
    
    // Recommended: Automatic detection
    const device = await Device.factory();
    
    // Optional: Manual handler selection
    const deviceWithHandler = await Device.factory({
      handlerName: 'Chrome111'
    });
  7. Basic usage of mediasoup-client

    v3

    To use mediasoup-client, you typically follow this lifecycle:

    1. Instantiate a Device.
    2. Load the device with router RTP capabilities retrieved from your signaling server.
    3. Create a Transport (Send or Receive) using parameters provided by the server.
    4. Set up event handlers for the transport (connect, produce, producedata) to communicate local parameters back to the server via your signaling layer.
    5. Use the transport to produce media tracks or data channels.
    import { Device } from 'mediasoup-client';
    import mySignaling from './my-signaling'; // Your signaling implementation
    
    // 1. Create a device
    const device = new Device();
    
    // 2. Load the device with router RTP capabilities from the server
    const routerRtpCapabilities = await mySignaling.request('getRouterCapabilities');
    await device.load({ routerRtpCapabilities });
    
    // 3. Check capabilities
    if (!device.canProduce('video')) {
    	console.warn('cannot produce video');
    }
    
    // 4. Create a transport
    const { id, iceParameters, iceCandidates, dtlsParameters, sctpParameters } =
    	await mySignaling.request('createTransport', {
    		sctpCapabilities: device.sctpCapabilities,
    	});
    
    const sendTransport = device.createSendTransport({
    	id,
    	iceParameters,
    	iceCandidates,
    	dtlsParameters,
    	sctpParameters,
    });
    
    // 5. Set up event handlers
    sendTransport.on('connect', async ({ dtlsParameters }, callback, errback) => {
    	try {
    		await mySignaling.request('transport-connect', {
    			transportId: sendTransport.id,
    			dtlsParameters,
    		});
    		callback();
    	} catch (error) {
    		errback(error);
    	}
    });
    
    sendTransport.on('produce', async ({ kind, rtpParameters, appData }, callback, errback) => {
    	try {
    		const { id } = await mySignaling.request('produce', {
    			transportId: sendTransport.id,
    			kind,
    			rtpParameters,
    			appData,
    		});
    		callback({ id });
    	} catch (error) {
    		errback(error);
    	}
    });
    
    // 6. Produce media
    const stream = await navigator.mediaDevices.getUserMedia({ video: true });
    const webcamTrack = stream.getVideoTracks()[0];
    const webcamProducer = await sendTransport.produce({ track: webcamTrack });
    
    // Produce data (DataChannel)
    const dataProducer = await sendTransport.produceData({
    	ordered: true,
    	label: 'foo',
    });
  8. Configure DataProducerOptions

    v3

    When working with a DataProducer, you can define its behavior using DataProducerOptions. This type allows you to specify how data is transmitted and attach custom application data.

    Available options:

    • ordered (boolean): Whether messages should be delivered in order.
    • maxPacketLifeTime (number): Maximum time in milliseconds for a packet to be considered alive.
    • maxRetransmits (number): Maximum number of retransmissions for a packet.
    • label (string): The label for the data channel.
    • protocol (string): The protocol for the data channel.
    • appData (DataProducerAppData): Custom application-specific data attached to the producer.
    type DataProducerOptions<DataProducerAppData extends AppData = AppData> = {
    	ordered?: boolean;
    	maxPacketLifeTime?: number;
    	maxRetransmits?: number;
    	label?: string;
    	protocol?: string;
    	appData?: DataProducerAppData;
    };
  9. Configure TransportOptions for initializing a Transport

    v3

    When creating a Transport instance, you must provide a TransportOptions object.

    Required Fields:

    • id: A unique string identifier for the transport.
    • iceParameters: An IceParameters object containing usernameFragment and password.
    • iceCandidates: An array of IceCandidate objects.
    • dtlsParameters: A DtlsParameters object containing fingerprints (an array of DtlsFingerprint).
    • direction: Either 'send' or 'recv'.

    Optional Fields:

    • sctpParameters: Configuration for SCTP data channels.
    • iceServers: An array of RTCIceServer objects.
    • iceTransportPolicy: An RTCIceTransportPolicy (e.g., 'all' or 'relay').
    • additionalSettings: A partial RTCConfiguration object.
    • appData: Custom application-specific data attached to the transport.
  10. Get RTCRtpReceiver stats with getStats()

    v3

    The getStats() method returns a Promise that resolves to an RTCStatsReport. This is used to retrieve real-time statistics for the associated RTCRtpReceiver.

    Throws:

    • InvalidStateError if the consumer is already closed.
    try {
      const stats = await consumer.getStats();
      console.log(stats);
    } catch (error) {
      if (error.name === 'InvalidStateError') {
        console.error('Cannot get stats: Consumer is closed');
      }
    }
  11. Initialize a Device using the Device constructor

    v3

    You can instantiate a Device directly using the new Device() constructor. Like the factory method, it can take DeviceOptions to specify a handlerName or handlerFactory. If no options are provided, it attempts to detect the device synchronously using detectDevice().

    Note: If automatic detection fails, the constructor will throw an UnsupportedError.

    import { Device } from 'mediasoup-client';
    
    // Synchronous instantiation with automatic detection
    const device = new Device();