DOCSEARCHbyAlgolia
Submit Docs

GPUPixel

repository·main·Indexed 25 days ago

https://github.com/pixpark/gpupixel

A high-performance, cross-platform C++11 library for applying image and video filters, with a specific focus on beauty effects using OpenGL/ES. It is compatible with iOS, Android, macOS, Windows, and Linux.

Tokens
43K
Snippets
104
Records
220
Agent score
81%

What's inside GPUPixel

  1. Overview of GPUPixel

    main

    GPUPixel is a real-time, high-performance image filter engine based on GPU. It is written in C++11 and utilizes OpenGL/ES to provide low-latency processing suitable for live streaming and WebRTC applications.

    Key Features

    • High Performance: Optimized for real-time use cases like live streaming.
    • Cross-Platform: Supports iOS, Android, Mac, Windows, and Linux.
    • Flexible Data Formats: Supports multiple input and output formats, including YUV and RGBA.
    • Filter Chaining: Includes built-in beauty filters (such as whitening and smoothing) that can be chained together to create complex visual effects.

  2. Introduction to GLFW

    main

    GLFW is an open-source, multi-platform library designed for developing applications using OpenGL, OpenGL ES, and Vulkan. It provides a platform-independent API for managing windows, creating contexts and surfaces, reading input, and handling events.

    Supported Platforms:

    • Windows
    • macOS
    • Linux (supports both X11 and Wayland)
    • Other Unix-like systems

    License: GLFW is licensed under the zlib/libpng license.

  3. Overview of libyuv functionality

    main

    libyuv is an open source library providing high-performance YUV scaling and conversion functionality. It is optimized for various hardware architectures including x86/x64 (SSSE3/AVX2), Arm (Neon/SVE2/SME), Mips (MSA), and RISC-V (RVV).

    Key capabilities include:

    • YUV Scaling: Preparing content for compression using point, bilinear, or box filters.
    • Webcam Conversion: Converting webcam formats to YUV for compression.
    • RGB Conversion: Converting YUV to RGB formats for rendering or effects.
    • Rotation: Rotating images by 90, 180, or 270 degrees to handle mobile device portrait modes.

  4. What is Dear ImGui?

    main

    Dear ImGui is a bloat-free, fast, and portable graphical user interface library for C++. It is designed for programmers to create content creation tools, visualization tools, and debug tools rather than consumer-facing UIs.

    Key characteristics:

    • Renderer Agnostic: It outputs optimized vertex buffers and command lists that you can render in your own 3D pipeline.
    • Self-contained: No external dependencies; you can simply add the .cpp and .h files from the root folder to your project.
    • Immediate Mode: It follows the IMGUI paradigm, which minimizes state synchronization and UI-related state storage on the user side.
    • Targeted Use Cases: Ideal for game engines, real-time 3D applications, embedded systems, and console platforms.

  5. Introduction to GPUPixel

    main
    GPUPixel is a high-performance, cross-platform image and video filter library designed with a small footprint. It is built using C++11 and OpenGL/ES, making it compatible with any platform that supports OpenGL/ES. It is primarily used for applying beauty filters and other visual effects to media.

  7. Manage mouse cursor latency and software rendering

    main

    Dear ImGui typically does not introduce significant lag, but there is a perceived difference between OS hardware-accelerated cursors and software-rendered UI content.

    If you want to visualize this difference or if you find the disconnect jarring, you can request Dear ImGui to draw a mouse cursor using the regular graphics API by enabling the io.MouseDrawCursor flag.

    Note: Rendering a software cursor at 60 FPS can feel sluggish. It is recommended to only enable software-rendered cursors during interactive drags rather than all the time.

  8. How libyuv handles CPU feature detection

    main
    libyuv contains multiple specialized implementations for various CPU architecture extensions. The library automatically detects the available hardware features at runtime and selects the most advanced architecture extension available for which a kernel implementation exists. This ensures optimal performance without requiring manual configuration by the user.

  9. Handle Gamma Correct Blending with FreeType

    main

    FreeType assumes blending occurs in linear space rather than gamma space. To achieve correct visual results when using FreeType, you must:

    1. Use sRGB.
    2. Convert to linear space in your pixel shader output.

    Note: Using FreeType will impact default Dear ImGui styles; you may need to tweak alpha values to compensate for the change in blending behavior.

  10. Understand LibYuv filtering modes and behavior

    main

    LibYuv uses different filtering modes for scaling (up/down sampling), each with distinct mathematical approaches to centering and sampling. The primary modes are:

    • Point: Uses a stepping rate of src_width / dst_width. In downsampling, it takes the middle pixel. In upsampling, it replicates pixels by starting at coordinate 0.
    • Bilinear: Uses a 2x2 pixel neighborhood. In downsampling, it centers on the middle 2x2 pixels. In upsampling, it stretches the image so the first and last source pixels map to the first and last destination pixels (note: this is noted as technically inconsistent with the downsampler).
    • Box: Averages the entire sampling box. In downsampling, it starts sampling at 0. For 2x downsampling, it is equivalent to Bilinear. In upsampling, it switches to Bilinear behavior.
    • Linear: A variation of bilinear sampling used in specific scaling implementations.

  11. Copyright and License Agreement for Contributors

    main

    By submitting code to the project, you agree to the following:

    • Your code will be distributed under the Dear ImGui license.
    • You grant all transferable rights to the project maintainer (including re-licensing, modifying, and distributing in source or binary forms).
    • You assign copyright to the project maintainer.
    • Important: Do not modify any existing copyright statements in files within your Pull Requests.

  12. Understand NV12 and NV21 biplanar formats

    main

    NV12 and NV21 are biplanar formats consisting of a full-sized Y plane followed by a single chroma plane containing interleaved UV values.

    • NV12: UV values are ordered U, V (weaved).
    • NV21: UV values are ordered V, U (weaved).

    Subsampling variations:

    • NV12/NV21: 4:2:0 subsampling (half width, half height chroma).
    • NV16: 4:2:2 subsampling (half width, full height chroma).
    • NV24: 4:4:4 subsampling (full width, full height chroma).