DOCSEARCHbyAlgolia
Submit Docs

Babit Multimedia Framework (BMF)

website·Indexed 30 days ago

https://babitmf.github.io/docs

A cross-platform, multi-language, customizable multimedia processing framework with strong GPU acceleration. BMF supports C++, Python, and Go interfaces, offering features such as dynamic graphs, GPU hardware transcoding via NVIDIA, and TensorRT-accelerated DL inference. It is fully compatible with FFmpeg and provides various execution modes including Sync, Generator, Preload, and PushData.

Tokens
36.7K
Snippets
234
Records
278
Agent score
74%

What's inside BMF

  1. Overview of BMF FFmpeg Compatibility

    BMF is fully compatible with FFmpeg's capabilities, allowing users in Python, C++, and Go to leverage FFmpeg's rich feature set for AV demuxing, decoding, filtering, encoding, and muxing through BMF's built-in modules and flexible API. It requires the FFmpeg library to be installed on the system.

  2. Overview of BabitMF (BMF) Core Concepts

    BabitMF (BMF) is a general video processing framework that uses a modular approach to manage atomic video processing capabilities. It organizes processing steps into a Directed Acyclic Graph (DAG) where data flows between modules.

    Key concepts include:

    • Graph: A DAG developed via the interface layer that represents a runnable pipeline or application.
    • Module/Node: An atomic video processing capability. A Module is the definition (which can be built-in or developed via SDK), and a Node is an instantiated Module.
    • Stream: The connection between nodes that defines the path of data transmission.
    • Packet: The universal data carrier in BMF. All data types (e.g., VideoFrame, AudioFrame) are encapsulated into Packets to be passed between modules.
    • Task: The scheduling unit used by the BMF engine to dispatch processing work to the process() entry of modules.

  3. Overview of Babit Multimedia Framework (BMF)

    BMF (Babit Multimedia Framework) is a cross-platform, multi-language, customizable multimedia processing framework developed by ByteDance. It allows developers to build high-performance multimedia processing links using graphs or pipelines, or by directly invoking individual processing capabilities. It is designed for scenarios such as video transcoding, frame extraction, enhancement, analysis, editing, video conferencing, and VR.

  4. Overview of BMF Dynamic Graph

    The Dynamic Graph feature in BMF allows developers to modify the graph structure (adding, removing, or configuring nodes) at runtime. This process is designed to occur without blocking other running nodes and with minimal performance penalty. To identify specific nodes for dynamic operations, an alias tag should be assigned to modules during graph construction.

  5. Overview of Babit Multimedia Framework (BMF)

    Babit Multimedia Framework (BMF) is a cross-platform multimedia processing framework designed for high-performance scheduling of DAGs, Graphs, and Pipelines. It is fully compatible with FFmpeg functions and standards, including demuxing, decoding, filtering, encoding, and muxing. BMF decouples processing algorithms from the project structure via a module-based system, allowing for high scalability and diversified collaboration.

  6. Overview of Babit Multimedia Framework (BMF)

    Babit Multimedia Framework (BMF) is a cross-platform, multi-language, and customizable multimedia processing framework developed by ByteDance. It is designed for high-scale production environments and is used for video streaming, live transcoding, cloud editing, and mobile pre/post processing.

  7. Overview of bmf_sdk::Future

    The bmf_sdk::Future class is used to track the readiness of data and manage synchronization between the CPU and device (e.g., GPU). It allows developers to record events and check if a result is ready before accessing it.

  8. Build BMF from source on macOS

    Building on macOS requires FFmpeg and specific dependencies based on architecture. For ARM (M1/M2), Python 3.9-3.12 is recommended. You must manually install binutils and libncurses as they are required for libbfd.
    # Install libncurses
./configure --prefix=/usr/local/opt/ncurses
make
sudo make install

# Install binutils for libbfd
wget https://ftp.gnu.org/gnu/binutils/binutils-2.43.1.tar.bz2
tar xvf binutils-2.43.1.tar.bz2
cd binutils-2.43.1
./configure --prefix=/usr/local/opt/binutils --enable-install-libiberty
make -j8
sudo make install

  9. Install BMF via pip

    BMF can be installed using pip. Python versions 3.6 to 3.12 are supported, though versions lower than 3.6.15 are not recommended as they may cause compilation failures. For users on x86_64 architecture with CUDA 12.2 installed, a GPU-specific version is available.
    # Standard installation
pip install BabitMF

# GPU version (x86_64 with CUDA 12.2)
pip install BabitMF-GPU

  10. BMF Architecture and Language Support

    BMF is organized into three layers to support multi-language development and scalability:

    1. Application Layer: Provides APIs in Python, Go, and C++ for project integration.
    2. Framework Layer: Handles graph/pipeline scheduling, cross-data type and cross-device data transfer backends, and SDKs for reformatting, color space conversion, and tensor operators.
    3. Module Layer: Contains atomic video processing capabilities. Modules can be developed independently in Python, Go, or C++, decoupling the algorithm implementation from the application layer.

  11. Use the Exception class for error handling in BMF C++

    The Exception class encapsulates detailed information about errors occurring within a BMF program. While it has a full constructor, it is typically not called explicitly by the developer; instead, exceptions are constructed and thrown implicitly using the BMF_Error() and BMF_Error_() macros.

  12. Create an AudioFrame instance

    The bmf_sdk::AudioFrame class can be instantiated using constructors or static make methods. You can create a frame by specifying the number of samples and layout, or by providing an existing TensorList of data.
    // Create an empty frame with specific samples and layout
bmf_sdk::AudioFrame frame1(int samples, uint64_t layout, bool planer = true, const TensorOptions &options = kUInt8);

// Create a frame from existing TensorList data
bmf_sdk::AudioFrame frame2(const TensorList &data, uint64_t layout, bool planer = true);

// Using static make methods
auto frame3 = bmf_sdk::AudioFrame::make(int samples, uint64_t layout, bool planer = true);
auto frame4 = bmf_sdk::AudioFrame::make(const TensorList &data, uint64_t layout, bool planer = true);