DOCSEARCHbyAlgolia
Submit Docs

CUDA Python

repository·main·Indexed 25 days ago

https://github.com/nvidia/cuda-python

A collection of packages providing access to the NVIDIA CUDA platform from Python, including high-level Pythonic abstractions and low-level C API bindings via cuda-bindings, cuda-core, and cuda-pathfinder.

Tokens
58.5K
Snippets
65
Records
413
Agent score
85%

What's inside cuda-python

  2. Overview of CUDA Python components

    main

    CUDA Python provides multiple ways to access the NVIDIA CUDA platform from Python, ranging from high-level Pythonic APIs to low-level C bindings. Key components include:

    • cuda.core: Idiomatic, Pythonic access to CUDA Runtime and core functionality.
    • cuda.bindings: Low-level Python bindings to CUDA C APIs.
    • cuda.pathfinder: Utilities for locating CUDA components in your Python environment.
    • cuda.compute: Python module for parallel algorithms (e.g., sort, scan, reduce) callable on the host.
    • numba-cuda-mlir: Evolution of Numba CUDA for JIT compilation and SIMT kernel development.
    • numba.cuda: Python DSL for the SIMT programming model.
    • cuda.tile: Python DSL for the CUDA Tile programming model (NumPy-like code in kernels).
    • nvmath-python: Pythonic access to NVIDIA CPU & GPU Math Libraries (Host, Device, and Distributed APIs).
    • nvshmem4py: Pythonic interface to the NVSHMEM library for PGAS programming.
    • Nsight Python: Profiling interface for performance analysis using NVIDIA Nsight Tools.
    • CUPTI Python: APIs for creating profiling tools via the CUDA Profiling Tools Interface.
    • Accelerated Computing Hub: Open-source learning materials for GPU computing.

  3. Overview of cuda.core

    main

    cuda.core provides a high-level, Pythonic interface to the CUDA runtime. Unlike cuda.bindings which provides 1:1 mappings to the CUDA driver and runtime APIs, cuda.core focuses on high-level constructs for common GPU tasks.

    Key capabilities include:

    • Compiling and launching CUDA kernels via JIT.
    • Asynchronous execution using CUDA graphs, streams, and events.
    • Multi-device coordination.
    • Memory management (allocation, transfer, and management).
    • Runtime linking of device code with Link-Time Optimization (LTO).

  4. Overview of cuda-pathfinder capabilities

    main

    The cuda.pathfinder module provides a centralized way to locate CUDA components.

    Supported features:

    • Locating and loading dynamic libraries (.so, .dll).
    • Locating CTK (CUDA Toolkit) header directories.

    Compatibility:

    • It is CUDA Toolkit (CTK) version-agnostic.
    • It follows the standard CUDA Toolkit support policy: the two most recent major versions are supported simultaneously.

  5. Understand the Handle and Object Registry pattern

    main

    The cuda-python library uses an Identity Map pattern (via two registries) to ensure that when raw CUDA driver handles (like CUgraphNode pointers) are returned to Python, they are mapped back to the original Python object instead of creating duplicate objects. This ensures object identity is preserved during driver round-trips.

    Key characteristics:

    • Weak References: Both registries use weak references so they do not prevent garbage collection/cleanup.
    • Cleanup: Entries are removed either explicitly (via destroy() or a Box destructor) or implicitly when the weak reference expires.
    • Identity Preservation: This prevents scenarios where querying the driver for an existing resource returns a new, distinct Python object that should have been the same instance as the original.

  7. Understand the Resource Handle Design in cuda.core

    main

    The cuda.core subpackage uses a specialized resource handle design to manage CUDA resources independently of Python's garbage collection. This ensures that CUDA resource lifetimes are managed correctly during interpreter shutdown or cross-language usage.

    Key architectural features include:

    • Separated Management: Resource management is handled in a layer independent of Python objects.
    • Structural Lifetimes: Lifetimes are encoded via embedded handle dependencies.
    • Dynamic Symbol Resolution: Uses a capsule to resolve CUDA driver symbols dynamically through cuda.bindings.
    • Efficient Accessors: To avoid the overhead of unnecessary Python object wrappers, handles provide overloaded accessors for different use cases instead of standard Python attributes.

  8. Use the cuda.bindings.cufile module

    main

    The cuda.bindings.cufile module provides Python bindings for the cuFile C APIs, enabling GPUDirect Storage capabilities.

    Requirements & Constraints:

    • Platform: Supported on Linux only.
    • Dependency: Requires NumPy (any recent 1.x or 2.x version).

    This module allows for high-performance I/O operations between GPU memory and storage.

  12. Use cuda.pathfinder to locate NVIDIA resources

    main

    The cuda.pathfinder module provides utilities for discovering and loading various NVIDIA components on your system. Use it to:

    • Load NVIDIA dynamic libraries.
    • Locate NVIDIA C/C++ header directories.
    • Find CUDA binary utilities.
    • Locate CUDA bitcode and static libraries.
    • Determine the CUDA path or home directory.