DOCSEARCHbyAlgolia
Submit Docs

PyAV Documentation

repository·main·Indexed 25 days ago

https://github.com/pyav-org/pyav

Pythonic bindings for FFmpeg's libraries, providing direct access to media via containers, streams, packets, codecs, and frames. PyAV is designed for developers requiring fine-grained control over media data and integration with NumPy and Pillow. It includes a CLI for inspecting library configurations, hardware devices, and supported codecs, as well as APIs for bitstream filtering, device enumeration, and metadata management via the av.Dictionary class.

Tokens
12.7K
Snippets
20
Records
123
Agent score
84%

What's inside PyAV

  1. Overview of PyAV capabilities

    main

    PyAV provides Pythonic bindings for FFmpeg, offering direct and precise access to media via containers, streams, packets, codecs, and frames. It is designed for developers who need fine-grained control over media data rather than simple command-line transformations.

    Key components exposed include:

    • libavformat: Container, Stream, and Packet objects.
    • libavcodec: Codec, CodecContext, BitStreamFilterContext, Frame, Plane, and Subtitle objects.
    • libavfilter: Filter and Graph objects.
    • libswscale: VideoReformatter.
    • libswresample: AudioResampler.
    • libavdevice: Accessible via specifying a format to containers.

  2. Use the Filter API to process media

    main

    PyAV provides a filtering system to manipulate media streams (e.g., scaling, cropping, or color conversion) using a graph-based architecture. The core components are:

    • Filter: Represents an individual filtering operation.
    • Graph: A collection of filters that defines the processing pipeline.
    • FilterContext: Manages the state and execution of the filter graph.
    • FilterLink: Defines the connections between different filters in the graph.

  3. Build PyAV from source on Windows

    main

    On Windows, use a Conda environment and the FFmpeg development files maintained by the PyAV project.

    git clone https://github.com/PyAV-Org/PyAV.git
cd PyAV
conda create --name pyav-dev --channel conda-forge python=3.11 cython setuptools numpy pillow pytest
conda activate pyav-dev
$ffmpegDir = Join-Path $env:CONDA_PREFIX "Library"
python scripts\fetch-vendor.py --config-file scripts\ffmpeg-latest.json $ffmpegDir
python setup.py build_ext --inplace --ffmpeg-dir=$ffmpegDir
python -m pytest

  4. Build PyAV on Windows

    main

    On Windows, you can create a development environment using Conda, fetch the necessary FFmpeg development files, and build PyAV. This process uses the same approach as the PyAV Windows continuous-integration build.

    conda create --name pyav-dev --channel conda-forge python=3.11 cython setuptools numpy pillow pytest
conda activate pyav-dev
$ffmpegDir = Join-Path $env:CONDA_PREFIX "Library"
python scripts\fetch-vendor.py --config-file scripts\ffmpeg-latest.json $ffmpegDir
python setup.py build_ext --inplace --ffmpeg-dir=$ffmpegDir
python -m pytest

  5. Build PyAV from source on Linux or macOS

    main

    To build PyAV from the latest source code on Linux or macOS, follow these steps to clone the repository, prepare a virtual environment, optionally build FFmpeg dependencies, and run the build.

    # Get PyAV from GitHub.
git clone https://github.com/PyAV-Org/PyAV.git
cd PyAV

# Prep a virtualenv.
source scripts/activate.sh

# Optionally build FFmpeg.
./scripts/build-deps

# Build PyAV.
make

  6. Install PyAV from source (Bring your own FFmpeg)

    main

    To compile PyAV against your own build of FFmpeg 8.x instead of using the bundled binary wheels, use the --no-binary flag with pip.

    PyAV requires the following FFmpeg libraries:

    • libavcodec
    • libavdevice
    • libavfilter
    • libavformat
    • libavutil
    • libswresample
    • libswscale

    Additionally, you must have pkg-config and Python's development headers installed.

    pip install av --no-binary av

  7. Manage timestamps during encoding

    main

    When encoding, follow these steps to ensure correct timing:

    1. Set Time Bases: Set av.CodecContext.time_base (ideally to the inverse of the frame rate). You may also set av.Stream.time_base as a hint to the muxer.
    2. Prepare Frame PTS: Prepare av.Frame.pts using the av.CodecContext.time_base.
    3. Rescale for Muxing: The encoded av.Packet.pts will be in the codec's time base. You must rescale the packet's PTS to the av.Stream.time_base before muxing, as stream operations assume packet time is in the stream's time base.

  8. Enable Hardware Acceleration for decoding and encoding

    main

    You can use hardware acceleration by passing an av.codec.hwaccel.HWAccel instance to PyAV functions.

    Hardware Decoding: Pass an HWAccel instance to av.open(). Frames are downloaded to system memory by default.

    import av
from av.codec.hwaccel import HWAccel

hwaccel = HWAccel(device_type="videotoolbox")
with av.open("input.mp4", hwaccel=hwaccel) as container:
    for frame in container.decode(video=0):
        ...

    Hardware Encoding: Pass an HWAccel instance to OutputContainer.add_stream(). Software frames passed to encode() will be uploaded to the device automatically.

    import av
from av.codec.hwaccel import HWAccel

with av.open("output.mp4", "w") as container:
    stream = container.add_stream(
        "h264_videotoolbox", 
        rate=30, 
        hwaccel=HWAccel(device_type="videotoolbox")
    )
    ...
    import av
from av.codec.hwaccel import HWAccel

# Decoding example
hwaccel = HWAccel(device_type="videotoolbox")
with av.open("input.mp4", hwaccel=hwaccel) as container:
    for frame in container.decode(video=0):
        ...

# Encoding example
with av.open("output.mp4", "w") as container:
    stream = container.add_stream(
        "h264_videotoolbox", 
        rate=30, 
        hwaccel=HWAccel(device_type="videotoolbox")
    )
    ...