Twilio Video JavaScript Library

repository·master·Indexed 20 days ago

https://github.com/twilio/twilio-video.js

A JavaScript SDK for integrating real-time voice and video capabilities into web applications. It provides tools to connect to video rooms, manage participants and tracks, and handle media capture via createLocalAudioTrack and createLocalVideoTrack. The library supports NPM installation, CDN usage, and TypeScript definitions.

Tokens
7.1K
Snippets
20
Records
38
Agent score
66%

What's inside twilio-video.js

  1. Configure Content Security Policy (CSP) for twilio-video.js

    master

    To ensure compatibility with twilio-video.js, configure your Content Security Policy with the following directives. Note that you may need to merge these with your existing policies.

    Standard Connection and Media:

    • connect-src wss://global.vss.twilio.com wss://sdkgw.us1.twilio.com
    • media-src mediastream:

    If loading from sdk.twilio.com:

    • script-src https://sdk.twilio.com

    If using Krisp Noise Cancellation with default-src self:

    • default-src self unsafe-eval
  2. Install twilio-video via CDN

    master

    You can include twilio-video.js directly in your HTML using a <script> tag. When using the CDN, the library is exposed via a global Twilio.Video object.

    <script src="//sdk.twilio.com/js/video/releases/2.35.0/twilio-video.min.js"></script>
  3. Handle browser autoplay policies for audio and video

    master

    Browsers (Chrome, Firefox, Safari) block autoplaying media unless the user has interacted with the page.

    For RemoteAudioTracks: In Safari, RemoteAudioTracks might be paused if no local media is being captured. You can handle this by checking if user interaction is required and providing a playback button.

    For RemoteVideoTracks:

    1. Ensure the user interacts with the application (e.g., clicks a 'Join' button) before calling Twilio.Video.connect().
    2. If joining on page load, set the muted attribute of the <video> element returned by VideoTrack.attach() to true.
    // Option 1: Ensure interaction before connecting
    document.getElementById('join_room').addEventListener('click', () => {
      Twilio.Video.connect(token, {
        name: 'my-room'
      });
    });
    
    // Option 2: Mute the video element to allow autoplay
    const video = videoTrack.attach();
    video.muted = true;
  4. Track hierarchy: LocalTrack and RemoteTrack

    master

    Twilio Video categorizes tracks based on whether they originate from the local participant or a remote participant:

    LocalTrack

    A track created by the local user. It can be one of the following:

    • LocalAudioTrack
    • LocalVideoTrack
    • LocalDataTrack

    RemoteTrack

    A track received from another participant in the room. It can be one of the following:

    • RemoteAudioTrack
    • RemoteVideoTrack
    • RemoteDataTrack

    DataTrack

    A specialized track type that can be either a LocalDataTrack or a RemoteDataTrack.

  5. Telemetry event payload structure

    master

    When the publish method is called, the payload object passed to your publisher will automatically include the following fields, merged with any custom data you might have provided via the SDK's internal emitters:

    • elapsedTime: The time in milliseconds since the provided connectTimestamp.
    • level: The severity level of the event ('info', 'warning', 'error', or 'debug').
    • group: The category of the event (e.g., 'network', 'quality').
    • name: The name of the event.
    • timestamp: The current timestamp when the event was emitted.
    • payload (optional): Any additional custom data associated with the event.
  6. Fix missing audio/video tracks in Angular applications on Safari

    master

    Angular applications using Zone.js may experience missing media tracks on Safari due to a misinteraction with RTCPeerConnection APIs. To fix this, include webapis-rtc-peer-connection.js from Zone.js in your application immediately after loading Zone.js.

    <script src="node_modules/zone.js/dist/zone.js"></script>
    <script src="node_modules/zone.js/dist/webapis-rtc-peer-connection.js"></script>
  7. Workaround for iOS 15 VideoTracks going black or page freezing

    master

    On iOS 15.1, certain interruptions (like incoming calls or backgrounding the browser) can cause VideoTracks to go black or the page to freeze. You can implement a shim that listens for pause and play events on the video element to intelligently re-attach the track.

    // Keeps track of video elements and their event listeners
    const videoElements = {};
    
    // Listen to onPlay and onPause events and intelligently re-attach the video element
    function shimVideoElement(track, el) {
      let wasInterrupted = false;
    
      const onPause = () => {
        wasInterrupted = true;
      };
    
      const onPlay = () => {
        if (wasInterrupted) {
          track.detach(el);
          track.attach(el);
          wasInterrupted = false;
        }
      };
    
      el.addEventListener('pause', onPause);
      el.addEventListener('play', onPlay);
    
      // Track this element so we can remove the listeners
      videoElements[el] = { onPause, onPlay };
    }
    
    // Apply the workaround after attaching the video element.
    videoTrack.attach(videoElement);
    shimVideoElement(videoTrack, videoElement);
    
    // Remove the listeners before detaching the video element.
    const { onPause, onPlay } = videoElements[videoElement];
    videoElement.removeEventListener('pause', onPause);
    videoElement.removeEventListener('play', onPlay);
  8. Fix iOS 15 echo issues after attaching VideoTracks

    master
    On iOS 15, attaching a VideoTrack to a video element can cause an echo if a LocalAudioTrack is already attached to an audio element. This is due to a bug where the audio element is unintentionally unmuted. To avoid this, skip attaching the LocalAudioTrack to an audio element.
  9. Fix iOS 15 low audio volume in Safari

    master

    Safari on iOS 15 may route audio to the earpiece instead of the speakers, resulting in low volume. A workaround is to pipe remote audio tracks into a single AudioContext and use a GainNode to increase the volume.

    // Make sure to reuse the audioContext object as browsers
    // have limits to the number of AudioContext instances you can create.
    const audioContext = new (window.AudioContext || window.webkitAudioContext)();
    
    function attachAudioTrack(remoteAudioTrack) {
      const audioNode = audioContext.createMediaStreamSource(new MediaStream([remoteAudioTrack.mediaStreamTrack]));
      const gainNode = audioContext.createGain();
    
      // Adjust this value depending on your customers' preference
      gainNode.gain.value = 20;
    
      audioNode.connect(gainNode);
      gainNode.connect(audioContext.destination);
    }
    
    // Attach the RemoteAudioTrack once received.
    attachAudioTrack(remoteAudioTrack);
  10. Note on audio output device changes (setSinkId)

    master
    The audioElement.setSinkId() method, used to change audio output devices, is only implemented in Desktop Chrome and Desktop Edge. In other browsers, users must change their audio output device via their operating system settings.