DOCSEARCHbyAlgolia
Submit Docs

mp4ff

repository·master·Indexed 20 days ago

https://github.com/eyevinn/mp4ff

A Go library and set of CLI tools for parsing and writing MP4 media files, with a focus on fragmented MP4 (fMP4) for streaming applications like DASH, HLS, and MSS. It supports AVC, HEVC, VVC, AV1, VP8, VP9, AAC, and AC-3, as well as WebVTT and TTML subtitles. The toolkit includes utilities for inspecting box hierarchies, extracting SPS/PPS, cropping progressive files, and performing on-the-fly encryption and refragmentation using cenc and cbcs schemes.

Tokens
7.6K
Snippets
24
Records
40
Agent score
68%

What's inside mp4ff

  1. Overview of mp4ff

    master
    The mp4ff module is a Go-based implementation for parsing and writing MP4 media files. It provides support for AVC and HEVC video, AAC and AC-3 audio, WebVTT (wvtt) and TTML (stpp) subtitles, and timed metadata tracks. While it can handle progressive MP4 files, it is primarily focused on fragmented MP4 files used in streaming protocols like MPEG-DASH, MSS, and HLS fMP4.

  2. How refragmentation works in stream-encrypt

    master

    Refragmentation allows the server to split large input fragments into smaller output fragments.

    • Mechanism: If an input fragment contains 60 samples and -samples 30 is set, the server produces two output fragments with 30 samples each.
    • Sequence Numbers: Sub-fragments maintain the same sequence number as their parent fragment, ensuring compatibility with streaming players.
    • Benefits: Enables lower latency and smaller chunk sizes suitable for adaptive bitrate streaming.
    • Implementation: Uses GetSampleRange() to fetch only the required samples for each sub-fragment, ensuring memory efficiency.

  3. Optimize I/O with SliceReader and SliceWriter

    master

    For high-performance scenarios, avoid the overhead of io.Reader (which often causes frequent allocations and copying). Instead, use the SliceReader and SliceWriter interfaces provided by the mp4ff/bits package.

    Using SliceReader

    Most boxes implement a Decode<X>SR(sr bits.SliceReader) method. For example, TrunBox provides DecodeTrunSR. Using these methods allows decoding directly from a slice of bytes, significantly reducing heap allocations.

    Using SliceWriter

    When encoding, using EncodeSW(sw bits.SliceWriter) directly on boxes provides better performance and fewer allocations compared to the standard Encode(w io.Writer) method.

  4. How encryption works in stream-encrypt

    master

    The server performs on-the-fly encryption of MP4 fragments during the streaming process.

    • Schemes: Supports cenc (AES-CTR) and cbcs (AES-CBC).
    • IV Derivation: Uses an incremental IV per fragment based on the fragment number.
    • Metadata: Automatically adds senc, saiz, and saio boxes to fragments. It also modifies the Init segment to convert sample entries to encv/enca and adds the sinf structure.
    • Implementation: Leverages mp4.InitProtect() and mp4.EncryptFragment() to process data without buffering entire fragments.

  5. Understand sample numbering conventions

    master

    The library follows the ISOBMFF standard regarding numbering:

    • One-based numbering: Sample numbers and other numeric arguments in functions and methods start at 1.
    • Zero-based storage: Internally, these are stored in slices using zero-based indexing. For example, sample number 1 corresponds to index 0 in the underlying slice.

  6. Understand the mp4.File structure and composition

    master

    The mp4.File type is the top-level structure for both progressive (non-fragmented) and fragmented MP4 files.

    Progressive (Non-fragmented) Files

    Attributes Ftyp, Moov, and Mdat point to their corresponding boxes.

    Fragmented Files

    A fragmented file can be an init segment, one or more media segments, or a combination (like a CMAF track). Key attributes include:

    • Init: Contains ftyp and moov boxes (CMAF header). May contain sidx boxes.
    • Segments: A slice of MediaSegment objects. Each segment starts with an optional styp box, zero or more sidx boxes, and one or more Fragments.
    • Fragment: Contains exactly one moof box followed by an mdat box (media data). It may contain one or more trun boxes (sample metadata) and one or more emsg boxes.

    Accessing Child Boxes

    Container boxes like MoovBox list all child boxes in a Children attribute. However, prominent child boxes are often exposed via direct named links for easier access.

    Warning: When using direct links, always assert that intermediate pointers are not nil to avoid panics.

  7. Create new fragmented MP4 files

    master

    Generating a fragmented file typically involves creating an init segment followed by media segments.

    1. Create the Init Segment

    Use mp4.CreateEmptyInit() and add tracks. You must fill in codec-specific parameters (like HEVC descriptors) into the sample descriptor.

    2. Produce Media Segments

    Media segments should use the same timescale set in the init segment. A segment contains one or more fragments. Each fragment consists of a moof and an mdat box.

    To create a segment from a collection of samples, use mp4.FullSample to define the data and metadata, then add them to a fragment within a segment.

    3. Encode the Segment

    Segments can be written to an io.Writer or a bits.SliceWriter.

    // 1. Create Init
init := mp4.CreateEmptyInit()
init.AddEmptyTrack(timescale, mediatype, language)
init.Moov.Trak.SetHEVCDescriptor("hvc1", vpsNALUs, spsNALUs, ppsNALUs)

// 2. Create Media Segment
seg := mp4.NewMediaSegment()
frag := mp4.CreateFragment(uint32(segNr), mp4.DefaultTrakID)
seg.AddFragment(frag)

for _, sample := range samples {
    frag.AddFullSample(sample)
}

// 3. Encode
err := seg.Encode(w) // to io.Writer
// or
err := seg.EncodeSW(sw) // to bits.SliceWriter

  8. Run the stream-encrypt HTTP server

    master

    The stream-encrypt tool is an HTTP streaming server that can encrypt and refragment MP4 files on-the-fly. You can run it using go run from the example directory.

    Basic Streaming

    To start a server that serves the default test file without encryption or refragmentation:

    go run *.go

    Once running, you can download the stream using curl:

    curl http://localhost:8080/enc.mp4 -o output.mp4
    go run *.go
curl http://localhost:8080/enc.mp4 -o output.mp4

  9. Install mp4ff CLI tools via Go

    master

    You can install specific tools directly from the repository using go install. Repeat this command for each tool you require.

    go install github.com/Eyevinn/mp4ff/cmd/mp4ff-info@latest
go install github.com/Eyevinn/mp4ff/cmd/mp4ff-encrypt@latest

  10. Encrypt MP4 streams with stream-encrypt

    master

    To serve encrypted MP4 files, you must provide a hex-encoded key, key ID, and IV, and specify an encryption scheme.

    Encryption Example

    This command configures the server to refragment files into 30-sample chunks and encrypt them using the cenc scheme:

    go run *.go \
  -samples 30 \
  -key 11223344556677889900aabbccddeeff \
  -keyid 00112233445566778899aabbccddeeff \
  -iv 00000000000000000000000000000000 \
  -scheme cenc

    Supported schemes are cenc (AES-CTR) and cbcs (AES-CBC).

  11. Configure stream-encrypt with input files and refragmentation

    master

    You can customize the server behavior using command-line flags to specify a different input file or to split fragments into smaller sizes (refragmentation).

    Using a Custom Input File

    go run *.go -input /path/to/your/video.mp4

    Refragmentation

    To split fragments into a specific number of samples (e.g., 30 samples per fragment) to reduce latency or chunk size:

    go run *.go -samples 30
    go run *.go -samples 30
curl http://localhost:8080/enc.mp4 -o refragmented.mp4

  12. Install mp4ff CLI tools on Linux (Debian, Fedora, Alpine)

    master

    Download the appropriate package (.deb, .rpm, or .apk) from the releases page and install it using your package manager:

    • Debian/Ubuntu: Use apt.
    • Fedora/RHEL: Use rpm.
    • Alpine: Use apk (note: --allow-untrusted may be required).

    For other distributions, download the .tar.gz archive and add the binaries to your PATH.

    sudo apt install ./mp4ff_<version>_linux_amd64.deb              # Debian/Ubuntu
sudo rpm -i mp4ff_<version>_linux_amd64.rpm                     # Fedora/RHEL
sudo apk add --allow-untrusted mp4ff_<version>_linux_amd64.apk  # Alpine