DOCSEARCHbyAlgolia
Submit Docs

CV-CUDA Documentation

repository·main·Indexed 25 days ago

https://github.com/cvcuda/cv-cuda

A high-performance, GPU-accelerated computer vision library for NVIDIA hardware designed to accelerate AI pipelines. It provides optimized image and video processing operators, including resizing, color conversion, and feature extraction, with multi-language API support for C, C++, and Python. Features include zero-copy interfaces for PyTorch, batching support for variable shapes, and specialized pre- and post-processing kernels.

Tokens
23.4K
Snippets
42
Records
185
Agent score
82%

What's inside CV-CUDA

  1. Overview of CV-CUDA application samples

    main

    CV-CUDA provides end-to-end deep learning pipeline samples that combine preprocessing, inference, and post-processing. Available application workflows include:

    • Hello World: A basic introduction to GPU-only image processing.
    • Image Classification: Demonstrates ResNet50 classification using TensorRT.
    • Object Detection: Demonstrates RetinaNet detection with bounding box visualization.
    • Semantic Segmentation: Demonstrates FCN-ResNet101 with artistic background effects.

  2. Overview of CV-CUDA 0.2.0-alpha features

    main
    CV-CUDA 0.2.0 is the first open-source release. It provides core components for Image and Tensor with batch support, and includes 25 operators that support variable shape batches. The library provides both C/C++ and Python APIs and supports interoperability with PyTorch-GPU, Pillow-CPU, and OpenCV-CPU.

  3. Overview of CV-CUDA features and capabilities

    main

    CV-CUDA is a GPU-accelerated library designed for high-throughput, cloud-scale image processing and computer vision pre- and post-processing pipelines.

    Key features include:

    • High-performance CV and image processing kernels.
    • Multi-language API support: C, C++, and Python.
    • Batching support for images with variable shapes.
    • Zero-copy interfaces for seamless integration with PyTorch.
    • Specialized kernels for both pre-processing (e.g., resizing, color conversion) and post-processing (e.g., Non-Maximum Suppression, Bounding Box drawing).

  4. Overview of CV-CUDA

    main

    CV-CUDA is an open-source library of GPU-accelerated computer vision algorithms designed for high-throughput, low-latency image and video processing. It is optimized for AI pipelines across NVIDIA cloud, desktop, and edge platforms.

    Key Features:

    • High-performance CV and image processing kernels written in CUDA.
    • Support for C, C++, and Python APIs.
    • Batching support with variable shape images.
    • Zero-copy interfaces to PyTorch and other Python frameworks.
    • Sample applications for classification, object detection, and image segmentation.

  5. Understand CV-CUDA Object Caching

    main

    CV-CUDA uses an internal resource management system to cache allocated objects for efficient reuse. This applies to Python objects like cvcuda.Image, cvcuda.Tensor, cvcuda.ImageBatchVarShape, and cvcuda.TensorBatch.

    Key behaviors:

    • Automatic Management: When a CV-CUDA allocated object goes out of scope, its memory is not released but stored in the cache for reuse.
    • Reuse Criteria: A new object will reuse cached memory if it has identical specifications (shape, data type, etc.).
    • Python vs C++: Only Python objects are cached; there is no C/C++ object caching.
    • Device Agnostic: CV-CUDA does not track which device the data resides on.
    • Memory Ownership:
      • Non-wrapped objects: Allocated by CV-CUDA; these increase the cache size.
      • Wrapped objects: Wrap externally-managed memory; these do not increase the cache size but still benefit from cache reuse logic.

  6. Use PyNvVideoCodec for hardware-accelerated video processing

    main

    PyNvVideoCodec provides Python bindings to NVIDIA's hardware-accelerated video codec APIs (NVDEC/NVENC). It is designed to work with CV-CUDA by decoding video directly to GPU memory, minimizing CPU involvement.

    Key capabilities include:

    • Hardware-accelerated decoding (e.g., H.264, NV12).
    • Direct decoding to GPU memory.
    • Batch frame decoding for high throughput.
    • Hardware-accelerated encoding for output.

    Typical use cases include video analytics preprocessing, real-time transformations, transcoding pipelines, and video quality enhancement.

  7. Use Tensor for N-dimensional arrays

    main

    A Tensor is an N-dimensional array with a uniform data type and layout. It is suitable for segmentation masks, feature maps, depth maps, and general numerical data.

    Key Properties:

    • Shape: Size of each dimension.
    • DataType: e.g., U8, F32.
    • TensorLayout: Semantic meaning of dimensions (e.g., NHWC, NCHW).
    • Strides: Byte offset between elements in each dimension.

    Standard Dimension Labels:

    • N: Batch/samples
    • C: Channels
    • H: Height
    • W: Width
    • D: Depth (3D spatial dimension)
    • F: Frames (temporal depth for video)

  8. New features in CV-CUDA v0.7.0-beta

    main

    CV-CUDA v0.7.0-beta introduces several enhancements for performance and hardware support:

    • Optimized Python bindings: Python calls now have near-zero overhead compared to C++ calls.
    • Label operator enhancements:
      • Added a masking option for conditional island removal.
      • Added support for signed 32bits output datatype.
    • Hardware Support: Added support for IGX Orin (utilizing dGPU, Ampere, or Ada RTX6000).

  9. Use the CV-CUDA Python API

    main

    The CV-CUDA Python API provides a high-level, Pythonic interface to CV-CUDA functionality via the cvcuda module. It is designed for high-performance computer vision tasks on NVIDIA GPUs and features zero-copy interoperability with major Python frameworks like PyTorch and CuPy.

    Key features include:

    • Zero-copy interoperability: Seamlessly work with PyTorch, CuPy, and other GPU-accelerated frameworks without expensive data transfers.
    • Pythonic interfaces: All CV-CUDA operators and data types are exposed through intuitive Python interfaces.

  10. Use the CV-CUDA C API

    main

    The CV-CUDA C API provides a low-level interface to all library functionality using C functions and types. It is organized into two main components:

    1. NVCV (Core Types): Provides fundamental data types and containers for computer vision applications.
    2. CV-CUDA (Operators): Provides high-performance, GPU-accelerated computer vision and image processing operators.

    Functions and types in this API use the NVCV or cvcuda prefixes (for example, NVCVImage or cvcudaResize).

    Note for other languages:

    • C++ developers: Use the C++ API for RAII wrappers and improved type safety.
    • Python developers: Use the Python API for equivalent functionality.

  11. Install full dependencies for all CV-CUDA samples

    main

    The minimal requirements files used in the Quick Start only provide enough packages for the hello_world sample. To run other samples involving operators, complex applications, or interoperability, use the provided installation script located in the samples directory.

    cd samples
./install_samples_dependencies.sh

  12. Best practices for CV-CUDA interoperability

    main

    When building pipelines that combine CV-CUDA with other libraries, follow these best practices:

    Memory Management

    • Zero-copy vs. Copying: Be aware of whether tensors share memory. When converting from CV-CUDA to PyTorch, use .clone() if you need to avoid shared buffers.
    • Buffer Lifetimes: Ensure CUDA buffers are not freed while other frameworks still hold references to them.

    Data Layout

    • Layout Conversion: CV-CUDA uses HWC (Height × Width × Channels) by default. PyTorch typically uses CHW (Channels × Height × Width). Use .permute() in PyTorch to convert layouts.
    • Explicit Layouts: When using cvcuda.as_tensor, be explicit about the layout (e.g., cvcuda.as_tensor(obj, "HWC")).

    Device and Performance

    • Device Consistency: Ensure all operations occur on the same GPU device.
    • Streams: Use appropriate CUDA streams for concurrent operations. Note that PyTorch, CuPy, and PyCUDA manage their own streams.
    • Minimize Transfers: Minimize CPU-GPU transfers and decode/encode directly to/from GPU memory using NvImgCodec or PyNvVideoCodec.
    • Batching: Use batch operations (e.g., PyNvVideoCodec batch decoding) for better performance.