DOCSEARCHbyAlgolia
Submit Docs

mp4box.js

repository·main·Indexed 25 days ago

https://github.com/gpac/mp4box.js

A JavaScript library for advanced MP4 file processing, including parsing, segmentation, and sample extraction. Compatible with both browser and Node.js environments, it supports progressive parsing, metadata inspection, and splitting MP4 files into segments for use with the Media Source Extensions (MSE) API. Version 2.4.1.

Tokens
10.9K
Snippets
15
Records
73
Agent score
81%

What's inside mp4box.js

  1. Overview of MP4Box.js features and use cases

    main

    MP4Box.js is a JavaScript library for advanced MP4 file processing, inspired by the GPAC MP4Box tool. It supports both browser and Node.js environments.

    Key Features

    • Progressive Parsing: Analyze MP4 content incrementally as it loads.
    • Segmentation: Split MP4 files into segments, specifically for use with the Media Source Extensions (MSE) API.
    • Sample Extraction: Extract media samples for tasks like audio analysis or subtitle generation.
    • Cross-Platform: Compatible with both browsers and Node.js.

    Common Use Cases

    • Metadata Inspection: Real-time inspection of MP4 metadata.
    • Adaptive Streaming: Segmenting files for MSE playback or adaptive streaming.
    • Feature Building: Extracting samples to build tools like timeline previews or TextTracks.

  3. Use MP4Box.js in the browser without a bundler

    main

    Since the library is written in ES6 modules, you can use it directly in the browser. You can either load a pre-built version via a CDN or use a local build.

    To use a local build, ensure you are using <script type="module"> in your HTML.

    <html
  <head>
    <meta charset="utf-8" />
    <title>MP4Box.js in the browser</title>
    <script src="https://cdn.jsdelivr.net/npm/mp4box/dist/mp4box.all.js"></script>
    <!-- Alternatively, you can use a local build -->
    <!-- <script type="module" src="mp4box.all.js"></script> -->
  </head>
  <body
    <script type="module">
      import * as MP4Box from './mp4box.all.js';

      // Create a new MP4Box instance
      const mp4box = MP4Box.createFile();

      // Example usage: Add a file to the MP4Box instance
      // Note: You would typically fetch or read a file here
      // For demonstration, we will just log the instance
      console.log(mp4box);
    </script>
  </body>
</html>

  4. Build MP4Box.js from source

    main

    MP4Box.js is modular to allow for flexible configuration and reduced bundle size. It is compiled using tsup into two main flavors:

    • all: Includes all features (parsing, writing, sample processing, fragmentation, etc.).
    • simple: Includes only basic box parsing (no writing or sample processing).

    Each flavor is built for ESM, CJS, and IIFE formats. To build the library from the source repository, install dependencies and run the build script.

    npm install
npm run build

  5. Use MP4Box.js with a bundler

    main

    If your project uses a modern bundler like Webpack, Rollup, or Vite, you can integrate MP4Box.js by importing it directly into your JavaScript or TypeScript files.

    import * as MP4Box from 'mp4box';

  6. Segment MP4 tracks into fragments

    main

    Segmentation allows you to split a track into Movie Fragments (typically for DASH/HLS streaming).

    Workflow:

    1. Create an MP4Box.ISOFile.
    2. In onReady, configure tracks using setSegmentOptions(track_id, user, options).
    3. Call initializeSegmentation(mode) to get the initial segments.
    4. Call start() to begin processing.
    5. Listen for segments via the onSegment callback.

    Segmentation Options:

    • nbSamples: Number of frames per segment (default: 1000).
    • rapAlignement: boolean (default: true). If true, segments start with a Random Access Point.
    • normalizeAudioSampleEntriesForMSE: boolean (default: true). Normalizes QuickTime audio for Media Source Extensions compatibility.

    Modes for initializeSegmentation(mode):

    • 'combined' (default): Returns a single initialization segment containing all configured tracks.
    • 'per-track': Returns one initialization segment per fragmented track.
    var mp4boxfile = MP4Box.createFile();
mp4boxfile.onReady = function(info) {
  mp4boxfile.onSegment = function (id, user, buffer, sampleNumber, last) {
    // buffer is an ArrayBuffer containing Movie Fragments
  };
  
  // Configure track 0 for segmentation
  mp4boxfile.setSegmentOptions(info.tracks[0].id, 'my-user-data', { nbSamples: 1000 });
  
  var initSegs = mp4boxfile.initializeSegmentation();
  mp4boxfile.start();
};

// Append data to trigger segmentation
mp4boxfile.appendBuffer(data);

  7. Extract individual samples from a track

    main

    To extract raw samples (frames) from a specific track, use the extraction API. This is similar to segmentation but returns individual samples instead of fragments.

    Workflow:

    1. In onReady, define a track/user identifier.
    2. Call setExtractionOptions(track_id, user, options).
    3. Call start().
    4. Listen for samples via the onSamples callback.

    Extraction Options:

    • nbSamples: Number of samples per callback (default: 1000).
    • rapAlignement: boolean (default: true). Ensures sample arrays start with a RAP.

    Sample Object Structure: Each sample in the samples array contains:

    • track_id: Number.
    • is_rap: boolean.
    • timescale: Number.
    • dts: Number.
    • cts: Number.
    • duration: Number.
    • size: Number.
    • data: ArrayBuffer (the actual media data).
    • description: String (Box description).

  8. Understand the MP4 hierarchy via Container Boxes

    main

    The MP4 file structure is a hierarchy of nested boxes. In MP4Box.js, this is modeled using ContainerBox subclasses. Common hierarchies include:

    • Movie Hierarchy: moovBox (Movie) $\rightarrow$ trakBox (Track) $\rightarrow$ mdiaBox (Media) $\rightarrow$ minfBox (Media Information) $\rightarrow$ stblBox (Sample Table).
    • Fragmented MP4 (fMP4) Hierarchy: moofBox (Movie Fragment) $\rightarrow$ trafBox (Track Fragment) $\rightarrow$ trunBox (Track Run).

    Each container box exposes its children as typed properties or arrays (e.g., moovBox.trak or trakBox.mdias).

  9. The Box class and its hierarchy

    main

    The Box class is the base class for all MP4 box structures in MP4Box.js. It manages the fundamental properties of an MP4 box, including its size, type (FourCC), and data.

    Key characteristics:

    • Type Identification: The box type is determined by a static fourcc property on the class or an instance-defined type string.
    • Nesting: Boxes can contain other boxes using the addBox method, which also maintains convenience arrays for specific box types (e.g., this.boxes or this.type + 's').
    • FullBox: A subclass of Box that includes additional header information: version and flags.
    • Data Management: Boxes hold their payload in a data property (either Array<number> or Uint8Array).

  10. Identify media sample entries by FourCC

    main
    MP4Box.js uses specific classes to represent different media sample entries in an MP4 file. Each class is associated with a unique 4-character code (FourCC) that identifies the codec or media type. When parsing a file, the library uses these FourCCs to instantiate the correct sample entry class (e.g., avc1 for H.264, mp4a for AAC, av01 for AV1).

  11. Understand the CompositionOffsetBox (ctts)

    main

    The ctts box (Composition Offset Box) is used in MP4 files to provide the offset between the Decoding Time Stamp (DTS) and the Presentation Time Stamp (PTS) for samples. This is essential for correctly rendering video frames that are displayed in a different order than they are decoded (e.g., B-frames).

    When using the unpack method on a cttsBox instance, the library automatically calculates the pts for each sample based on its dts and the offsets stored in the box.

    Key properties:

    • sample_counts: An array of integers indicating how many samples share the same offset.
    • sample_offsets: An array of integers representing the offset value applied to the samples in the corresponding sample_counts group.

  12. Understand the Time-to-sample (stts) box

    main

    The sttsBox (TimeToSampleBox) is used in MP4 files to define the time increments between consecutive samples. It maps a sequence of sample counts to specific time deltas (increments). This information is critical for calculating the Decoding Time Stamps (DTS) of samples within a track.

    Key properties:

    • sample_counts: An array of integers representing how many consecutive samples share the same time delta.
    • sample_deltas: An array of integers representing the time increment (in units of timebase) for the corresponding group of samples in sample_counts.