DOCSEARCHbyAlgolia
Submit Docs

MoQ (Media over QUIC)

repository·main·Indexed 23 days ago

https://github.com/moq-dev/moq

A next-generation live media protocol designed for real-time latency at massive scale. MoQ leverages QUIC for prioritization and partial reliability to provide a generic pub/sub transport for media and live data. The project includes the obs-moq plugin for OBS Studio, ergonomic Go bindings (moq-go and moq-go-ffi), web demos for inspecting and publishing streams, and tools for managing MoQ Internet-Drafts and package repository infrastructure.

Tokens
175.2K
Snippets
422
Records
963
Agent score
77%

What's inside MoQ

  1. Overview of Rust Core Libraries

    main

    The MoQ Rust implementation provides the reference implementation of the protocol. Key core libraries include:

    • moq-net: The networking layer. Negotiates either moq-lite or full moq-transport. Handles broadcasts, tracks, groups, frames, QUIC stream management, and backpressure.
    • hang: A media-specific encoding/streaming library built on moq-net. Includes a catalog for track discovery and supports formats like H.264/265, VP8/9, AV1, AAC, and Opus.
    • moq-mux: Media muxers/demuxers for importing formats like fMP4/CMAF, MPEG-TS, and FLV into MoQ.
    • moq-token: JWT authentication library for HMAC and RSA/ECDSA signing and path-based authorization.
    • web-transport: QUIC and WebTransport implementation (based on Quinn).
    • moq-native: Helpers for configuring Quinn QUIC endpoints and TLS.

  2. Overview of moq-msf

    main

    The moq-msf crate provides implementations for the MOQT Streaming Format (MSF) catalog types. It is designed to handle catalog formats as defined in draft-ietf-moq-msf-01, with extended support for CMAF packaging via draft-ietf-moq-cmsf-00.

    Key features include:

    • Version Agnosticism: The Catalog type acts as a version-agnostic snapshot of tracks. While the wire format handles version and indirection for initialization data (initDataList/initRef), the consumer interacts with a unified representation.
    • Compatibility: The parser accepts both draft-00 and draft-01 catalogs. Serialization always emits the newest draft, ensuring compatibility with older publishers.
    • Legacy Support: Fields from draft-00, such as generatedAt and isComplete, are preserved as root catalog fields.
    • Use Case: Primarily used by moq-mux for the muxing and demuxing of media.

  3. Overview of primary MoQ implementations (Rust and TypeScript)

    main

    MoQ has two primary implementations that serve as the foundation for other language bindings:

    Rust (Native)

    The reference implementation used by all server-side tools and the FFI core. Key crates include:

    • moq-net: Real-time pub/sub.
    • hang: Media catalog and container.
    • moq-mux: Import/export for fMP4/CMAF, MPEG-TS, and FLV.
    • moq-native: QUIC endpoint helpers.

    TypeScript (Web)

    Designed for the browser, utilizing WebTransport, WebCodecs, and WebAudio. Key packages include:

    • @moq/net: Real-time pub/sub.
    • @moq/hang: Media library.
    • @moq/watch: Subscribe and render functionality.
    • @moq/publish: Capture and publish functionality.

  4. Overview of libmoq

    main

    libmoq provides C bindings for moq-net via FFI (Foreign Function Interface). It is designed to enable MoQ integration in C/C++ applications and other languages that can interface with C.

    Key features include:

    • C API: Header files for C integration.
    • FFI bindings: A safe Rust-to-C interface.
    • Build system integration: Support for CMake and pkg-config.

  5. Overview of moq-net

    main

    moq-net

    moq-net is the Rust networking layer for Media over QUIC (MoQ). It provides real-time pub/sub capabilities with built-in caching, fan-out, and prioritization, all built on top of the QUIC protocol.

    Protocol Negotiation

    During session setup, moq-net automatically negotiates between two wire protocols:

    1. moq-lite: The simplified protocol (default).
    2. moq-transport: The full IETF protocol.

    This negotiation ensures compatibility with any moq-transport CDN.

    Core Abstractions

    MoQ media is organized into a hierarchy of data structures:

    • Broadcasts: Discoverable collections of tracks.
    • Tracks: Named streams of data, which are subdivided into groups.
    • Groups: A sequential collection of frames (typically beginning with a keyframe).
    • Frame: A timed chunk of data.

  6. Overview of MoQ Rust crates

    main
    The MoQ project provides several Rust crates for implementing the Media over QUIC protocol, ranging from networking and authentication to media processing and FFI bindings. Use these crates to build native applications that support pub/sub media streaming over QUIC or WebTransport.

  7. Overview of TypeScript MoQ Libraries

    main

    The TypeScript implementation enables MoQ in web browsers using WebTransport and WebCodecs. The ecosystem is divided into core networking, media handling, and utilities.

    Core Libraries

    • @moq/net: The core pub/sub transport protocol for browsers. Implements the moq-lite specification using WebTransport-based QUIC. Supports broadcasts, tracks, groups, and frames.
    • @moq/hang: A high-level media library using Web Components and WebCodecs for streaming audio and video.

    Media Packages

    • @moq/watch: Provides a JavaScript API and <moq-watch> Web Component to subscribe to and render MoQ broadcasts.
    • @moq/publish: Provides a JavaScript API and <moq-publish> Web Component to publish media to MoQ broadcasts.

    Utilities

    • @moq/signals: A reactive signals library used for state management.
    • @moq/clock: Utilities for timestamp synchronization.
    • @moq/token: JWT token generation and verification for browsers.

  8. Overview of moq-nvenc

    main

    moq-nvenc provides Rust bindings for the NVIDIA Video Codec SDK (NVENC for encoding and NVDEC for decoding). It is designed to be vendored within the MoQ workspace to support hardware-accelerated H.264/H.265 encoding and decoding on Linux.

    Key characteristics:

    • Runtime Loading: It uses dlopen to load driver libraries (libnvidia-encode for NVENC and libnvcuvid for NVDEC) at runtime. This means binaries can be built and started on machines without NVIDIA drivers or GPUs (where they will fall back to other encoders).
    • Build Independence: The build process does not require the CUDA toolkit or NVIDIA driver libraries to be present. It is compatible with GPU-less build environments.
    • Cross-Platform Compilation: The sys bindings are plain C-ABI definitions, allowing the crate to compile on any platform, including macOS. However, moq-video only invokes the NVENC path on Linux; on other platforms, it acts as a compile-only stub.

  9. Overview of moq-boy

    main

    moq-boy is a crowd-controlled Game Boy Color emulator designed to stream video and audio using the Media over QUIC (MoQ) protocol.

    Key features include:

    • Low Latency: Viewers watch the stream with sub-second latency.
    • Crowd Control: Viewers connect via the @moq/boy web client to collectively send button inputs back to the emulator.
    • Auto-Pause: The emulator automatically pauses when there are no active viewers.

    For orchestration details, including the justfile and ROM hosting setup, refer to the demo repository.

  10. Overview of moq-mux capabilities

    main

    moq-mux provides media muxers and demuxers to convert existing media formats into MoQ broadcasts. It is primarily used for ingesting content into the MoQ ecosystem by converting traditional formats into hang broadcasts.

    Supported Container Formats:

    • fMP4/CMAF: Fragmented MP4 and Common Media Application Format.
    • MPEG-TS: Transport stream (supports both import and export).
    • Matroska / WebM: EBML container (supports both import and export).
    • FLV: Flash Video / RTMP container (supports both import and export). Includes legacy H.264 + AAC + MP3 and enhanced-RTMP FourCC codecs (HEVC, AV1, VP9, Opus, AC-3, E-AC-3) with multitrack support.
    • Annex B: Raw H.264/H.265 NAL unit streams.

  11. Overview of moq-native

    main

    moq-native is a helper library designed for building native Media over QUIC (MoQ) applications. It acts as a high-level abstraction layer on top of moq-net, providing managed connectivity, security, and lifecycle management.

    Key responsibilities include:

    • Transport Management: Establishing MoQ connections across multiple transport protocols.
    • Security: Handling TLS and certificate generation.
    • Reliability: Implementing reconnection logic.
    • Infrastructure: Providing logging setup and clap-compatible configuration for CLI binaries.

  12. Overview of MoQ (Media over QUIC)

    main

    Media over QUIC (MoQ) is a next-generation live media protocol designed to transmit media concurrently using QUIC to avoid latency build-up during network congestion.

    Key characteristics include:

    • Adaptive: Supports real-time, interactive, and lean-back experiences within a unified stack.
    • Scalable: Content can be cached and fanned-out via CDNs (e.g., Cloudflare).
    • Extensible: Allows custom tracks for various live content types.
    • Modern Web Integration: Uses WebTransport, WebCodecs, and WebAudio APIs.
    • Cross-Platform: Native Rust and TypeScript implementations, with FFI bindings for C, Python, Kotlin, Swift, and Go.
    • Secure: Encrypted via TLS and authenticated via JWT.
    • Backwards Compatible: Supports CMAF and HLS for legacy devices.
    • Decentralized: Supports self-hosted CDNs and P2P connectivity via Iroh.