HLS.js

repository·master·Indexed 12 days ago

https://github.com/video-dev/hls.js

A JavaScript library that enables HTTP Live Streaming (HLS) playback in browsers using HTML5 video and MediaSource Extensions (MSE). It transmuxes MPEG-2 Transport Stream and AAC/MP3 streams into ISO BMFF (MP4) fragments, supporting VOD, Live playlists with DVR, Low-Latency HLS, and fragmented MP4 (fMP4). Features include adaptive bitrate (ABR) switching, DRM support via EME, and subtitles in WebVTT, CEA-608/708, and IMSC1 formats.

Tokens
65.6K
Snippets
148
Records
238
Agent score
97%

What's inside HLS.js

  1. What is HLS.js

    master

    HLS.js is a JavaScript library that implements an HTTP Live Streaming (HLS) client. It works directly on top of a standard HTML <video> element by utilizing HTML5 video and MediaSource Extensions (MSE).

    Key technical details:

    • Transmuxing: It transmuxes MPEG-2 Transport Stream and AAC/MP3 streams into ISO BMFF (MP4) fragments. This process is performed asynchronously using a Web Worker when available.
    • Format Support: Supports both MPEG-2 TS and HLS + fMP4.
    • Language: Written in ECMAScript6 and TypeScript, transpiled to ECMAScript5.
  2. Overview of hls.js internal architecture and controllers

    master

    hls.js is organized into specialized controllers that manage different aspects of the HLS lifecycle:

    • Stream & Buffer Management: stream-controller ensures the buffer is filled according to quality logic; buffer-controller manages SourceBuffer operations (append, flush, reset).
    • Quality & ABR: abr-controller manages bitrate-based quality switching; level-controller handles manifest loading and level switching; cap-level-controller adjusts quality based on player dimensions.
    • Track Management: audio-track-controller and audio-stream-controller manage alternate audio tracks; subtitle-track-controller and subtitle-stream-controller handle subtitle fragments and decryption.
    • Data Processing: transmuxer (often running in a Web Worker) demuxes TS/AAC/MP3 into MP4 boxes; mp4-remuxer converts these into fragmented ISO BMFF compatible with MediaSource.
    • Metadata: id3-track-controller manages ID3 metadata; timeline-controller handles CEA-708 caption data.
  3. Core Features of HLS.js

    master

    HLS.js provides a wide range of playback features including:

    • Playlist Support: VOD & Live playlists with DVR support and Low-Latency HLS (Partial Segments, Blocking Playlist Reload, etc.).
    • Container Support: Fragmented MP4 (HEVC, AV1, VP9, Dolby Vision, etc.) and MPEG-2 TS (H.264, H.265, AAC, MP3, AC-3, etc.).
    • Security: AES-128, AES-256, and AES-256-CTR decryption; "identity" format SAMPLE-AES (MPEG-2 TS only); and EME support for DRM (FairPlay, PlayReady, Widevine).
    • Adaptive Streaming: Manual & Auto Quality Switching with three modes:
      • Instant switching: Immediate switch at current position.
      • Smooth switching: Switch at the next loaded fragment.
      • Bandwidth conservative switching: Switch at the next fragment without flushing the buffer.
    • Subtitles & Captions: WebVTT, CEA-608/708, and IMSC1 (TTML) support.
    • Advanced Playback: I-frame trick-play, accurate seeking (not limited to keyframes), and HLS Interstitials.
    • Analytics: Built-in monitoring for Network and Video events, playback session metrics, and Common Media Client Data (CMCD).
  4. How ABR (Adaptive Bitrate) switching works

    master

    The abr-controller determines the optimal quality level using a bitrate-based algorithm. It monitors fragment loading speed by tracking the stats.loaded counter from the fragment loader.

    To prevent erratic switching, it uses two Exponential Weighted Moving Averages (EWMA):

    • Fast EWMA: Adapts quickly to bandwidth drops to switch to lower quality levels rapidly.
    • Slow EWMA: Prevents ramping up to higher quality levels too quickly when bandwidth increases.

    The final bandwidth estimate is the minimum of these two averages. If the 'expected time of fragment load completion' exceeds the 'expected time of buffer starvation' and the time needed for the next quality level, the current fragment load may be aborted via a FRAG_LOAD_EMERGENCY_ABORTED event.

  5. Use I-Frame variants for secondary video rendering

    master

    I-Frame variants (HLS #EXT-X-I-FRAME-STREAM-INF) can be used to load video I-Frames into a secondary HTMLVideoElement. This is useful for synchronized frame rendering.

    To use them, call hls.createIFramePlayer() which returns an HlsIFramesOnly instance. This instance uses the current HLS instance's iframeVariants as its levels.

    Key behaviors:

    • I-Frame instances do not respond to external seeking or currentTime changes on the attached element. You must use loadMediaAt(time) to buffer and seek.
    • The playlist selection is driven by the video element's dimensions if capLevelToPlayerSize: true is set in the config. Ensure the element is sized before calling startLoad() or loadMediaAt().
    • Audio in muxed segments is dropped; only video I-Frames are buffered.
    • An I-Frame is considered appended on FRAG_BUFFERED and rendered on the seeked event of the HTMLVideoElement.
    const mainVideo = document.getElementById('video_1');
    const iframeVideo = document.getElementById('video_2');
    const hls = new Hls();
    
    let hlsIframesOnly: HlsIFramesOnly | null = null;
    
    hls.loadSource('http://example.com/primary.m3u8');
    hls.attachMedia(mainVideo);
    
    hls.once(Events.INIT_PTS_FOUND, createHlsIframesOnlyIfNeeded);
    
    function createHlsIframesOnlyIfNeeded() {
      if (hls.url !== hlsIframesOnly?.url) {
        hlsIframesOnly = null;
      }
      if (!hlsIframesOnly && hls.iframeVariants.length) {
        hlsIframesOnly = hls.createIFramePlayer();
        if (hlsIframesOnly) {
          hlsIframesOnly.attachMedia(iframeVideo);
          hlsIframesOnly.startLoad();
          
          hlsIframesOnly.once(
            Events.LEVEL_UPDATED,
            (name, { details: { fragments } }) => {
              /* fragments contains all iframe start times and durations */
            },
          );
          hlsIframesOnly.on(Events.FRAG_BUFFERED, (name, { frag }) => {
            /* iframe buffered */
          });
          hlsIframesOnly.on(Events.ERROR, (name, { error }) => {
            if (error.name == 'QuotaExceededError') {
              /* MSE buffer is full */
            }
          });
        }
      }
    }
    
    function renderIFrame(currentTime) {
      iframeVideo.onseeked = () => null;
      hlsIframesOnly?.loadMediaAt(currentTime);
    }
  6. Check HLS.js browser compatibility

    master

    HLS.js requires browsers that support the MediaSource Extensions (MSE) API with video/MP4 mime-type inputs.

    Supported Browsers

    • Chrome: 47+ (Desktop), 5+ (Android)
    • Firefox: 51+ (Desktop), 5+ (Android)
    • Edge: Windows 10+
    • Safari: 10+ (macOS 10.11+), iPadOS 13+, iOS 17.1+ (using Managed Media Source)

    Distribution Variants

    • UMD (dist/hls.js, dist/hls.min.js, etc.): Embeddable via <script> tag or require(). Includes an inlined transmuxer Web Worker. Targets the browser list above with an ES2016 runtime baseline.
    • ESM (dist/hls.mjs, etc.): Intended for modern bundlers. Uses ES2015+ syntax but stays below ES2019.

    Important for ESM users: The ESM builds do not bundle the transmuxer Web Worker. You must manually point workerPath to a worker file to avoid running transmuxing on the main thread.

    const hls = new Hls({
      workerPath: 'https://cdn.jsdelivr.net/npm/hls.js@1/dist/hls.worker.js',
    });
  7. HLS.js Build Flavors: Full vs Light

    master

    HLS.js can be built in different flavors to optimize for size or feature set.

    light build limitations

    The hls.light.*.js files are smaller but exclude the following:

    • Features: Alternate-audio, subtitles, CMCD, EME (DRM), Variable Substitution, Interstitials, I-frame trick-play, Media Capabilities, or MPEG-2 TS advanced codecs (HEVC and AC-3).
    • Controllers/Interfaces: AudioStreamController, AudioTrackController, CuesInterface, EMEController, SubtitleStreamController, SubtitleTrackController, TimelineController, CMCDController, InterstitialsController, InterstitialsManager, IFrameController, HlsIFramesOnly, and HlsImageIFramesOnly.

    Note: Content Steering is included in the light build.

  8. Understand how hls.js handles buffer holes and playback stalls

    master

    The stream-controller monitors playback progress. If the playhead stops moving for longer than config.highBufferWatchdogPeriod (and the video is not ended, paused, or seeking), hls.js attempts to recover:

    1. Jump over buffer holes: If a known malformed fragment is detected, hls.js will seek to the beginning of the next playable buffered range.
    2. Nudge currentTime: hls.js will attempt to nudge the currentTime until playback recovers. It retries every second and will report a fatal error after reaching config.maxNudgeRetry retries.

    Note: A 500ms buffer threshold is used internally to account for browser behavior near the end of buffered ranges. Holes often occur during stream discontinuities or quality level switches.

  9. Understand Track and TrackSet structures

    master

    HLS tracks are represented by the Track interface (extending BaseTrack), which may include a buffer or initSegment. A TrackSet groups these tracks by type, such as audio, video, or audiovideo.

    export interface Track extends BaseTrack {
        buffer?: SourceBuffer;
        initSegment?: Uint8Array<ArrayBuffer>;
    }
    
    export interface TrackSet {
        audio?: Track;
        audiovideo?: Track;
        video?: Track;
    }
  10. Understand the hls.js architectural design principles

    master

    The architecture of hls.js is based on a modular subsystem model designed for decoupled communication. Key principles include:

    • Subsystem Modularization: Main functionalities are partitioned into several distinct subsystems.
    • Centralized Instantiation: All subsystems are instantiated and managed by the primary Hls instance.
    • Event-Driven Communication: Subsystems rely heavily on events for both internal coordination and external communication with the consumer.
    • EventEmitter3: The library uses eventemitter3 for high-performance event handling.
    • Bundling: The project is bundled for browser environments using rollup.
  11. Using HLS.js in Node.js (SSR)

    master
    HLS.js is designed for browser environments. You can safely require the library in a Node.js runtime for Server-Side Rendering (SSR) purposes; it will export a dummy object and will not throw an error. However, HLS.js is not instantiable in Node.js.
  12. Handle errors with ErrorController and ErrorActionFlags

    master

    The ErrorController manages error states and recovery. When an error occurs, you can use ErrorActionFlags to determine how the player should react.

    Common flags include:

    • None: Do nothing.
    • SwitchToSDR: Switch to a Standard Dynamic Range stream.
    • ResetMediaSource: Reset the MediaSource.
    • MoveAllAlternatesMatchingKey: Attempt to move to an alternate stream matching the current key.
    // ErrorActionFlags enum values
    // None = 0
    // MoveAllAlternatesMatchingHost = 1
    // ResetMediaSource = 16
    // SwitchToSDR = 8
    // MoveAllAlternatesMatchingKey = 4