Wasp HLS

repository·main·Indexed 41 days ago

https://github.com/peaberberian/wasp-hls

A high-performance HLS media streaming engine for the web (v0.5.0) that leverages a Rust-based WebAssembly core and Web Workers to offload heavy I/O and parsing from the main UI thread. It utilizes Media Source Extensions (MSE) for playback and supports Adaptive BitRate (ABR), MPEG-2 TS to fMP4 transmuxing, and various HLS playlist tags for VoD, Live, and Event content.

Tokens
40.1K
Snippets
135
Records
198
Agent score
51%

What's inside Wasp HLS

  1. Overview of project scripts

    main

    The scripts directory contains helper utilities for building, testing, generating, and serving the project. The scripts are organized into the following functional areas:

    • tasks: High-level task entry points, typically invoked via npm scripts.
    • utils: Shared helper functions used across various scripts.
    • packager: Tools used to generate local test HLS (HTTP Live Streaming) content.

    Additionally, the directory includes standalone scripts for specific operations like WASM ABI checks, embedded asset generation, and local server utilities.

  2. Overview of Wasp-hls

    main

    Wasp-hls is an HLS (HTTP Live Streaming) media player library for the web. It is designed as a streaming engine rather than a full UI application.

    Key characteristics include:

    • WebAssembly Core: The core logic is written in Rust and compiled to WebAssembly for optimal performance and explicit memory management, avoiding JavaScript Garbage Collection (GC) pressure.
    • Web Worker Execution: The engine runs primarily in a Web Worker. This offloads heavy I/O and parsing tasks from the main thread, preventing UI lag from affecting playback and vice-versa. This is particularly beneficial for low-latency streaming.
    • MSE Integration: It utilizes the Media Source Extensions™ (MSE) API to push media chunks into the browser's <video> element via SourceBuffers.

    Note: This project is a proof of concept and is currently under heavy development.

  3. Overview of wasp-hls capabilities and roadmap

    main

    wasp-hls is an HLS media player library that supports various streaming scenarios including VoD, Live, and Event content. It utilizes WebWorkers and WebAssembly to offload heavy logic and transmuxing from the main thread, and leverages Media Source Extensions (MSE) for playback.

    Key Features

    • Adaptive BitRate (ABR): Throughput-based variant selection, automatic codec filtering, and smart/fast-switching logic.
    • Request Scheduling: Lazy playlist downloading, live content refreshing, buffer goal implementation, and parallel audio/video loading.
    • Media Demuxing: Supports MPEG-2 Transport Streams, Fragmented MPEG-4 (fMP4), and AAC audio. It can transmux MPEG-2 TS to fMP4 in JavaScript for platforms like Chrome or Firefox.
    • MSE & Buffer Management: Handles multiple simultaneous buffers (audio/video), discontinuity gaps, and browser Garbage Collection detection.
    • Track Management: Provides an API to set audio tracks.

    Current Limitations & Roadmap

    • Low-Latency: Proper support for HLS low-latency content is pending.
    • Subtitles: WebVTT and IMSC subtitles are not yet handled.
    • Video Tracks: API for video track selection is planned.
    • DRM: Content decryption is currently not supported.
  4. Explore the WaspHlsPlayer API

    main

    The WaspHlsPlayer API provides a comprehensive set of features for managing HLS playback. This documentation serves as the authoritative reference for all public variables and methods.

    Note: Only the variables and methods explicitly defined in the official API documentation are guaranteed to be stable. Any other properties or methods encountered may change without notice and are not considered part of the public API contract.

  5. Explore the demo application source code

    main

    The demo directory contains the source code for the demonstration application. The application is built using React and serves as a reference for how to integrate the player.

    Key entry points and structures:

    • Entry Point: index.tsx initializes the React application, renders the player UI, and attaches debugging helpers to the global window object.
    • components/: Contains the React components used to build the demo's user interface.
    • utils/: Contains small helper functions used exclusively within the demo application.
  6. WaspHlsPlayer overview

    main

    The WaspHlsPlayer is a media player library designed for efficient HLS (HTTP Live Streaming) playback on the web. It achieves high performance by offloading media loading, parsing, and buffering operations to a WebWorker and using WebAssembly for optimized processing.

    This architecture ensures that heavy UI interactions do not interfere with media buffering and that media processing does not cause UI lag. The player supports MSE-in-Workers on compatible browsers for even better performance.

  7. Understand the structure of the `wasm` directory

    main

    The wasm directory serves as the integration point between WebAssembly and JavaScript. It contains:

    • The compiled WebAssembly target files.
    • Handwritten JavaScript glue code located in the ./js subdirectory, used to link the Wasm module to the JavaScript environment.
    • The ./abi directory, which contains files used for synchronized code generation across both Rust and JavaScript (e.g., creating synchronized enums and bindings).
  8. Convert playlist time to media time or JavaScript Dates

    main

    The value returned by getSeekableMinimumPosition is in playlist time (seconds). To work with this value in other contexts, use the following methods:

    • Media Time Conversion: To convert between media time and playlist time, use the getMediaOffset method to find the offset.
    • Date Conversion: If the content timeline uses EXT-X-PROGRAM-DATE-TIME, you can convert the playlist time into a JavaScript Date object using the positionToDate method.
  9. Understand playlist time vs media time

    main

    The getMediaDuration method returns time in playlist time (in seconds).

    Playlist time is the time extrapolated from the MediaPlaylist. For example, in live content, this might be the Unix timestamp corresponding to when the media was broadcasted. This differs from media time, which is the time associated with the underlying HTML media element (e.g., the currentTime attribute of an HTMLMediaElement).

    In WaspHlsPlayer, the API defaults to playlist time for consistency. To convert between media time and playlist time (for example, to use standard HTML properties), use the getMediaOffset method to find the offset between the two.

  10. Understand the wasp-hls architecture and component structure

    main

    The wasp-hls library is a multi-layered HLS player architecture designed to offload heavy media logic from the main thread to a WebWorker to ensure smooth playback and minimize rebuffering.

    Core Layers

    • JS API (ts-main): The primary entry point for developers. It runs on the Main Thread and handles interactions with the browser's HTMLMediaElement and the user interface.
    • Web Worker (ts-worker & rs-core): The engine of the player. It runs in a background thread to prevent UI lag from affecting media buffering.
      • TypeScript Logic (ts-worker): Manages communication between the Main Thread (via postMessage) and the WebAssembly core.
      • Rust Core (rs-core): The high-performance playback logic compiled to WebAssembly (Wasm). It handles decision-making, event reaction, and playback supervision.
    • Transmuxer (ts-transmux): A TypeScript-based mpeg2ts to fmp4 transmuxer used by the worker to prepare media for buffering.

    Data Flow and Communication

    1. Main Thread $\leftrightarrow$ Web Worker: Communication occurs via postMessage calls to the MessageReceiver (defined in ts-worker/MessageReceiver.ts).
    2. Web Worker $\leftrightarrow$ WebAssembly: The TypeScript worker uses TS bindings (defined in ts-worker/bindings.ts) to call WebAssembly functions and provide the Rust core with access to Web APIs (like MSE) that are otherwise unavailable in a worker.
    3. WebAssembly $\leftrightarrow$ JS Bindings: The Rust core interacts with JavaScript through Rs bindings (defined in rs-core/bindings/).
  11. When to use the WASM ABI manifest

    main

    The manifest should be used for functions that meet these criteria:

    • Their raw ABI signature is part of the stable JS/WASM contract.
    • Their TypeScript or Rust declarations are repetitive.
    • Their documentation needs to stay synchronized across generated outputs.

    Note: You should keep higher-level marshalling logic handwritten if the function:

    • Packs or unpacks domain-shaped payloads.
    • Conditionally allocates memory.
    • Requires complex control flow that would make the manifest behave like a DSL.

    Examples of functions that use generated signatures but manual marshalling include appendBuffer, inspectSegment, and __web_event__playback_tick.

  12. How Wasp-hls handles HLS streaming

    main

    Wasp-hls implements the logic required to consume HLS content. The workflow is as follows:

    1. Load Multivariant Playlist: The library fetches the central .m3u8 file (the Master Playlist) which contains the characteristics of the content (video qualities, audio tracks, subtitles, etc.).
    2. Analyze Requirements: It detects the desired characteristics based on parameters like bandwidth, quality, codecs, preferred language, and accessibility.
    3. Fetch Media Segments: It requests the specific small chunks of media data corresponding to the chosen quality and tracks.
    4. Communicate via MSE: It pushes these chunks to the browser using the Media Source Extensions (MSE) API so the browser can decode and play the media.

    This architecture allows for efficient seeking, selective loading of tracks (e.g., only loading the desired audio track), and support for live streaming.