DOCSEARCHbyAlgolia
Submit Docs

Intel® Video Processing Library (Intel® VPL)

repository·main·Indexed 18 days ago

https://github.com/intel/libvpl

A hardware-accelerated video processing library providing a unified API for video decode, encode, and processing on Intel® GPUs via a dispatcher mechanism. The documentation includes instructions for building the VPL specification, as well as guides for building and running API 1.x (legacy-decode, legacy-encode, legacy-vpp) and API 2.x (hello-decode, hello-decvpp) samples on Linux and Windows.

Tokens
87.7K
Snippets
199
Records
430
Agent score
61%

What's inside Intel VPL

  1. Overview of Intel® Video Processing Library (Intel® VPL)

    main

    Intel® Video Processing Library (Intel® VPL) is a programming interface designed for video decoding, encoding, and processing. It enables developers to build portable media pipelines that run on CPUs, GPUs, and other accelerators.

    Key capabilities include:

    • Device Discovery and Selection: Facilitates finding and choosing appropriate hardware for media-centric and video analytics workloads.
    • Zero-Copy Buffer Sharing: Provides API primitives for efficient data handling between different components of a media pipeline.
    • Portability: Designed to be backwards and cross-architecture compatible, allowing optimal execution on current and future hardware without requiring source code changes.

  2. Overview of the `legacy-vpp` sample

    main

    The legacy-vpp sample demonstrates how to use the Intel® VPL 1.35 common API to perform video resizing.

    Functionality:

    • Takes a raw native format video elementary stream as input.
    • Resizes the video to a target resolution (default 640x480).
    • Writes the output to out.raw in BGRA raw video format.

    Default Configuration:

    • Target device: GPU
    • Input format: NV12
    • Output format: BGRA
    • Output resolution: 640 x 480

  3. Overview of the vpl-infer sample

    main

    The vpl-infer sample demonstrates an inference pipeline that combines Intel® Video Processing Library (Intel® VPL) and Intel® Distribution of OpenVINO™ Toolkit. The pipeline follows a decode -> resize -> infer workflow.

    Pipeline Workflow:

    1. Decode: Decodes an H.265 video elementary stream using Intel® VPL.
    2. Resize: Resizes the decoded frames using Intel® VPL.
    3. Infer: Performs object detection on each frame using the OpenVINO™ toolkit.

    Default Configuration:

    • Target device: GPU
    • Input video format: H.265 video elementary stream
    • Input IR network model: Object detection
    • Output: Class ID, Bounding Box Location, and Confidence Score

  4. What is Intel® Video Processing Library (Intel® VPL)?

    main

    Intel® Video Processing Library (Intel® VPL) provides hardware-accelerated video decode, encode, and processing capabilities on Intel® GPUs. It is designed to support AI visual inference, media delivery, cloud gaming, and virtual desktop infrastructure use cases.

    To use Intel® VPL, you must install at least one implementation:

    • Intel® VPL GPU Runtime: For use on Intel® Iris® Xe graphics and newer.
    • Intel® Media SDK: For use on legacy Intel® graphics.

  6. Understand the `hello-sharing-ocl` sample

    main

    The hello-sharing-ocl sample demonstrates how to use the Intel® VPL 2.X common API to perform video processing with OpenCL surface sharing. It specifically covers two workflows:

    1. Encoding with OpenCL sharing: Taking a 2D OpenCL image and encoding it to an H.265 video elementary stream.
    2. Decoding with OpenCL sharing: Decoding an H.265 video stream and exporting mfxFrameSurface1 to a 2D OpenCL image.

    Target Environment:

    • OS: Windows 10
    • Hardware: Intel® GPU (compatible with Intel® VPL GPU implementation via graphics drivers).
    • Native Formats: Uses NV12 for raw frames.

  7. Intel® VPL Function Reference Overview

    main

    The Intel® Video Processing Library (Intel® VPL) provides a comprehensive set of APIs for video processing tasks. The library is organized into several functional modules:

    • Video Decoding: APIs for decoding compressed video streams.
    • Video Encoding: APIs for encoding raw video into compressed formats.
    • Video VPP (Video Post Processing): APIs for image processing tasks like scaling, color conversion, and filtering.
    • Video Core: Fundamental video processing primitives and core logic.
    • Session Management: APIs for managing VPL sessions and hardware contexts.
    • Memory Management: APIs for allocating and managing video and system memory.
    • Implementation Capabilities: APIs to query the capabilities and supported features of the underlying hardware/implementation.
    • Adapters: APIs for interfacing with different backends or hardware drivers.
    • Video Decode/VPP: Combined workflows for decoding and post-processing.

  8. What is Surface Sharing in Intel® VPL?

    main

    Introduced in API 2.10, Surface Sharing allows for the direct exchange of video surfaces between the Intel® VPL runtime and external frameworks or graphics APIs. This avoids the overhead of mapping data to system memory and copying it via mfxFrameSurfaceInterface::Map.

    There are two primary directions for surface sharing:

    1. Importing: Enables Intel® VPL to use raw video data from an external source as input for encoding or Video Post-Processing (VPP) operations.
    2. Exporting: Enables an application to access raw video data produced by Intel® VPL (from decoding or VPP operations) directly, without requiring a system memory map.

    Requirement: This functionality is only supported when using the internal memory management model.

  10. Camera Raw Acceleration Processing Structures

    main
    Intel® VPL provides a set of specialized structures designed for Camera Raw Acceleration Processing. These structures allow developers to configure various image processing stages typically required when handling raw sensor data, such as white balance, color correction, noise reduction, and lens distortion correction.

  11. Understand ChromaSiting in mfxExtColorConversion

    main

    When using mfxExtColorConversion for color conversion, the ChromaSiting field indicates the chroma location based on the input and output chroma formats.

    VPP InputVPP OutputChromaSiting Indicates
    MFX_CHROMAFORMAT_YUV420MFX_CHROMAFORMAT_YUV444Chroma location for input
    MFX_CHROMAFORMAT_YUV420MFX_CHROMAFORMAT_YUV422Chroma location for input
    MFX_CHROMAFORMAT_YUV422MFX_CHROMAFORMAT_YUV444Chroma location for input
    MFX_CHROMAFORMAT_YUV420MFX_CHROMAFORMAT_YUV444Chroma location for input
    MFX_CHROMAFORMAT_YUV444MFX_CHROMAFORMAT_YUV420Chroma location for output
    MFX_CHROMAFORMAT_YUV444MFX_CHROMAFORMAT_YUV422Chroma location for output
    MFX_CHROMAFORMAT_YUV420MFX_CHROMAFORMAT_YUV420Chroma location for input and output
    MFX_CHROMAFORMAT_YUV420MFX_CHROMAFORMAT_YUV422Horizontal location for input and output, vertical location for input

  12. Discover Intel VPL implementation capabilities

    main

    Intel VPL implementations may expose different subsets of encoders, decoders, VPP filters, color formats, and memory types. You can discover these capabilities using two primary methods:

    1. Before or after session creation: Use MFXEnumImplementations to get a list of supported implementations, including their supported features (encoders, decoders, VPP filters, etc.). Note that the legacy MSDK implementation does not support this method.
    2. After session creation: Use the specific Query functions (e.g., MFXVideoENCODE_Query, MFXVideoDECODE_Query, MFXVideoVPP_Query) to obtain low-level implementation capabilities for a specific session.

    If an implementation provides any video processing features, MFXQueryImplsDescription must return a valid mfxImplDescription structure containing the implementation's capabilities.

    // Example conceptual flow for capability discovery
// 1. Enumerate implementations
MFXEnumImplementations(...);

// 2. Create session and query specific capabilities
MFXVideoENCODE_Query(session, ...);