DOCSEARCHbyAlgolia
Submit Docs

Microsoft Media Foundation

repository·master·Indexed 41 days ago

https://github.com/microsoft/media-foundation

A collection of sample applications, tooling, and documentation for implementing media pipelines and using Media Engine on Windows. Includes guidance on performance tracing with WPR, analyzing traces with Media Experience Analyzer (MXA), and implementing custom sources using IMFMediaEngineExtension, IMFMediaSource, and IMFMediaStream.

Tokens
8.2K
Snippets
10
Records
44
Agent score
51%

What's inside Media Foundation

  2. Overview of clearkeyStoreCDM sample

    master

    The clearkeyStoreCDM solution is a sample designed to demonstrate how to load a Content Decryption Module (CDM) using a Universal Windows Platform (UWP) application. It consists of two primary components:

    1. A native DLL: Implements the clearkey CDM.
    2. A UWP application: Contains the logic to specify and load the DLL (currently configured to load clearkeyDLL).

  3. Understand MediaEngineEMEUWPSample capabilities and limitations

    master

    The MediaEngineEMEUWPSample is a C++ UWP application demonstrating protected Playready EME (Encrypted Media Extensions) media playback using the MediaFoundation MediaEngine API and WinRT composition APIs. It is designed for efficient media playback in UWP applications requiring low-level UI rendering control and custom logic for fetching/demuxing content.

    Platform Support:

    • Windows Desktop
    • Xbox
    • HoloLens
    • IoTCore

    ⚠️ Important Limitation: This sample uses APIs that are currently outside the app API partition. While these are planned to move to the app API partition in the 20H2 SDK release, applications using these specific APIs cannot be submitted to the Windows Store until that change is implemented.

  4. Implement a Basic Synchronous MFT

    master

    A basic MFT can be implemented by providing a class that implements the IMFTransform interface. The provided BasicMFT sample acts as a synchronous passthrough.

    Key implementation details for a passthrough MFT:

    • ProcessInput: Caches the input sample to be used as output.
    • ProcessOutput: Makes a copy of the cached sample to the output.
    • GetInputStreamInfo: Should set MFT_INPUT_STREAM_PROCESSES_IN_PLACE.
    • GetOutputStreamInfo: Should set MFT_OUTPUT_STREAM_PROVIDES_SAMPLES.

    For full interface details, refer to the IMFTransform documentation.

  5. How the Media Engine Custom Source pipeline works

    master

    This sample demonstrates a Media Foundation pipeline that uses a XAML swapchain panel and a Media Engine with a MF Custom Source.

    Core Components and Workflow

    • MF Custom Source: Provides greater control over the data passed to the Media Engine. It allows the source to respond directly to Media Engine events and requests such as start, seek, pause, stop, and shutdown.
    • Source Reader: Retrieves samples from the source file and decodes them. These samples are passed to the custom source when the Media Engine requests more data.
    • Media Engine Pipeline: Processes the samples (typically passing them through audio or video decoders to the respective renderers).
    • XAML Swapchain Panel: During setup, the video renderer duplicates a composition surface handler and passes it to the XAML swapchain panel. This mechanism enables the Media Engine's video renderer to present directly to the XAML application.

  6. Understand the XAML Swapchain Media Pipeline

    master

    The samples demonstrate a media pipeline designed for video playback using a XAML swapchain panel. The architecture follows this flow:

    1. Source: The raw media file.
    2. Source Reader: Gets a sample from the source and decodes it.
    3. MFT (Media Foundation Transform): Acts as a processor (e.g., decoding or ML effects). In the basic 'XAML Sample', the MFT is removed and the Source Reader passes samples directly to the swapchain.
    4. Swapchain: Presents the processed samples on the screen.

    Key Components in the XAML Sample implementation:

    • InitializeGraphics: Creates a D3D device and swapchain for composition.
    • DoPresentationLoop: The main playback loop that initializes components and presents frames to the swapchain panel until the end of the stream.

  7. How MediaEngine and WinRT Composition work together for efficient playback

    master

    This sample demonstrates a high-performance architecture for UWP media playback by combining the IMFMediaEngine API with the WinRT composition visual layer.

    Core Components

    • MediaEngine (IMFMediaEngine): Encapsulates a highly efficient Media Foundation pipeline. It leverages hardware acceleration for video decoding, processing, and rendering. It includes optimizations like deep video frame queueing/batching, audio offload, and MMCSS (Multimedia Class Scheduler Service) tagging for efficient CPU utilization.
    • WinRT Composition APIs: Used to compose the video rendered by the MediaEngine within the application window. This allows for low-level control over UI rendering and layering.

    Integration Workflow

    To integrate the MediaEngine's video output into the application's visual tree, the following pattern is used:

    1. Create a Composition Swapchain: The application renders UI elements using low-level D3D APIs and presents to a D3D11 swapchain. This swapchain must be a composition swapchain, created via IDXGIFactoryMedia::CreateSwapChainForCompositionSurfaceHandle.
    2. Associate with a Surface: The swapchain is associated with a CompositionSurface.
    3. Apply to Visual Tree: The CompositionSurface is set on a CompositionSurfaceBrush, which is then associated with a CompositionVisual in the visual tree.
    4. Manage Layering: The application manages the visual tree to control the position and layering of visual elements (including the video) within the window.

  8. Capture Media Foundation traces for Application/Browser Video Playback

    master

    To capture traces for debugging video playback issues in applications or browsers, use the mftracelog.exe utility.

    Steps:

    1. Download and Setup:
      • Download mftracelog.zip from the latest releases.
      • Unzip the file and locate the folder corresponding to your machine's architecture (amd64, x86, or arm64).
    2. Configure and Start:
      • Close all instances of the video playback application or browser.
      • Launch the appropriate mftracelog.exe.
      • In the UI, select the Simple and Full Traces radio buttons.
      • Click Start Tracing.
    3. Reproduce:
      • Reproduce the issue. It is recommended to complete the reproduction in less than 2 minutes to prevent excessively large trace files.
    4. Stop and Collect:
      • Click Stop Tracing.
      • File Explorer will open showing the generated output files.
      • Select all generated files, right-click, and choose "compress to zip file" to prepare them for sharing.

  12. Install the clearkeyStoreCDM sample

    master

    To install the built application on a Windows 10 machine (or VM):

    1. Copy the entire App Package output directory to the target machine.
    2. Open an Administrator PowerShell window in the App Package directory.
    3. Execute the script: Add-AppDevPackage.ps1.
    4. Sign the native DLL located at: C:\Program Files\WindowsApps\\*clearkeyStoreCDM*.