TurboRes Apple ProRes Video Decoder

TurboRes supports two multithreading models:

1. Shared-memory multithreading: The fastest option. Multiple threads operate on the same memory to decode a single packet. Requires cross-origin isolation in browsers (see COOP/COEP headers).
2. Worker pool-based multithreading: Multiple workers decode different packets in parallel. This has higher latency due to message passing.

Configuration Options

• useSharedMemory: Set to true for shared-memory multithreading.
• concurrency: Number of threads to use. Defaults to navigator.hardwareConcurrency. Set to 0 to force synchronous decoding on the main thread.

Browser Requirements for Shared Memory

To use useSharedMemory: true in a browser, you must set one of the following sets of HTTP response headers:

Option A (Standard):

• Cross-Origin-Opener-Policy: same-origin
• Cross-Origin-Embedder-Policy: require-corp

Option B (More permissive, but not supported in Safari):

• Cross-Origin-Opener-Policy: same-origin
• Cross-Origin-Embedder-Policy: credentialless

// Enable shared-memory multithreading with 8 threads
const decoder = await Decoder.create({
    useSharedMemory: true,
    concurrency: 8,
});

// Force synchronous decoding
const syncDecoder = await Decoder.create({
    useSharedMemory: true,
    concurrency: 0,
});

You can compare the decoding performance of TurboRes, ffmpeg.wasm, and native FFmpeg using the provided benchmark files. All benchmarks measure the time taken to decode a specific video file ten times to determine decoding speed.

Web-based Benchmarks (TurboRes and ffmpeg.wasm)

Use the following HTML files for web-based testing:

• benchmark-turbores.html
• benchmark-ffmpeg-wasm.html

Important: When running these HTML benchmarks, ensure your browser console is closed, as an open console can slow down WASM execution and skew results.

Native FFmpeg Benchmark

Use the shell script to benchmark native FFmpeg via the CLI:

./benchmark-ffmpeg-native.sh -t <threads> [-a] <video-file>

Flags:

• -t <threads>: Specify the number of threads to use.
• -a: Enable hardware acceleration. Note: This requires running on a Mac with VideoToolbox available.

./benchmark-ffmpeg-native.sh -t 8 -a input.mov

To decode ProRes media, create a Decoder instance and use it to decode individual packets into Frame objects. TurboRes uses an "errors as values" pattern, so you should check if the returned result is an instance of Error.

1. Create a Decoder: Use Decoder.create() with configuration options like proresFourCc (e.g., 'apch') and useSharedMemory.
2. Create a Frame: Instantiate a new Frame() to hold the output data.
3. Decode: Call decoder.decode(packetData, frame) where packetData is a Uint8Array containing a single ProRes frame.
4. Cleanup: Always call frame.clear() and decoder.close() to free resources.

import { Decoder, Frame } from 'turbores';

// Create a decoder
const decoder = await Decoder.create({
    proresFourCc: 'apch',
    useSharedMemory: true,
});
if (decoder instanceof Error) {
    // Handle the error...
}

// Create a frame to hold the output data
const frame = new Frame();

const result = await decoder.decode(
    packetData, // Uint8Array containing a single ProRes frame
    frame,
);
if (result instanceof Error) {
    // Handle the error...
}

// When done, clean up
frame.clear();
decoder.close();

The Decoder class provides mechanisms to manage resources and prevent memory exhaustion during high-throughput decoding.

Backpressure and Queue Management:

• decodeQueueSize: Returns the number of decoding tasks currently queued but not yet finished. Monitor this to apply backpressure to your packet source.
• desiredSize: Returns how many more packets can be queued before reaching the internal _highWaterMark. Keep queuing while this value is positive to maximize performance.
• dequeued: A promise that resolves whenever a packet finishes decoding. Use this to synchronize with the decoder's progress.

Resource Cleanup:

• close(): Closes the decoder and releases internal resources. It returns a promise that resolves once all currently queued decodes are complete. This method is idempotent.
• The decoder also supports the Disposable and AsyncDisposable interfaces (using Symbol.dispose and Symbol.asyncDispose).

TurboRes uses two different runtime models for managing workers and memory, depending on the performance requirements and environment capabilities.

SharedMemoryRuntime

Designed for high-performance scenarios using WebAssembly shared memory. It initializes a single WebAssembly.Memory object that is shared across all workers. This allows for zero-copy data access but requires careful management of thread-local state (TLS).

• Use getSharedMemoryRuntime() to access the singleton instance.
• Use ensureWorkers(count) to scale the number of workers.

MessagePassingRuntime

A more traditional model where workers communicate via message passing. This is useful when shared memory is not desired or available.

• Use getMessagePassingRuntime() to access the singleton instance.
• Use registerDecoder(decoderId, bitDepth, allowedOutputFormats) to inform workers about available decoders.
• Use ensureWorkers(count) to scale the number of workers.

When decoder.decode() succeeds, it returns a FilledFrame object containing the decoded pixel data and metadata.

Key properties include:

• frameData: Uint8Array containing the pixel data.
• pixelFormat: The format string (e.g., 'I422P10').
• scanType: 'progressive' or 'interlaced'.
• codedWidth / codedHeight: The dimensions of the encoded packet.
• visibleWidth / visibleHeight: The dimensions of the visible area.
• pixelAspectRatio: An object with { num, den }.
• colorPrimaries, colorTransfer, colorMatrix: Integer identifiers for color metadata.
• colorRangeFull: Boolean indicating if the color range is full.

Note: The frame's data is valid until it is used for a new decoding task. To keep multiple frames, create multiple Frame instances.

// Example of result properties
result.frameData; // => Uint8Array
result.pixelFormat; // => 'I422P10'
result.scanType; // => 'progressive'
result.codedWidth; // => 1920
result.codedHeight; // => 1088
result.visibleWidth; // => 1920
result.visibleHeight; // => 1080
result.pixelAspectRatio; // => { num: 1, den: 1 }
result.colorPrimaries; // => 9 (ITU-R BT.2020)
result.colorTransfer; // => 18 (ITU-R BT.2100-2 (HLG))
result.colorMatrix; // => 9 (ITU-R BT.2020)
result.colorRangeFull; // => false