CUDA Core Compute Libraries (CCCL)

cudax (CUDA Experimental) is a library providing experimental features that are currently in development. While these features are subject to change and lack stability guarantees, they are intended for users to experiment with for specific high-performance use cases.

Key capabilities provided by cudax include:

• Asynchronous host-to-device and device-to-host byte-wise mdspan copies.
• Byte fill operations for mdspan.
• Uninitialized storage support.
• Graph functionality.
• Dimensions description functionality.
• places for managing execution and data affinity across multiple devices.
• An implementation of the STF (Sequential Task Flow) programming model.

CUB (CUDA Unbound) provides reusable software components for every layer of the CUDA programming model. It offers both scalar interfaces (single thread operations) and collective interfaces (operations entered simultaneously by a group of threads).

CUB's components include:

• Parallel primitives:
  • Thread-level: Reduction and other primitives specialized for specific CUDA architectures.
  • Warp-wide: "Collective" primitives like cooperative prefix scan and reduction.
  • Block-wide: "Collective" primitives for cooperative I/O, sort, scan, reduction, and histogram, compatible with arbitrary thread block sizes.
  • Device-wide: Parallel sort, prefix scan, reduction, and histogram, compatible with CUDA dynamic parallelism.
• Utilities: Fancy iterators, thread/block I/O, PTX intrinsics, and device/kernel/storage management.

NVRTCC is a tool designed to simplify NVRTC (NVIDIA Runtime Compilation) testing. It mimics the nvcc compilation trajectory but replaces the nvcc-generated PTX code with code compiled via NVRTC. This allows developers to keep source files nearly identical to standard nvcc source files while leveraging NVRTC. NVRTCC ensures necessary symbols are present in the generated PTX, allowing nvcc to handle host compilation and linking.

Note: This is an internal, experimental tool and is not intended for general public use.

CUDA Core Compute Libraries (CCCL) is a unified repository of three essential CUDA C++ libraries designed to provide high-performance, easy-to-use C++ abstractions for CUDA developers:

• Thrust: A high-level C++ parallel algorithms library. It enhances productivity and enables performance portability between GPUs and multicore CPUs via configurable backends (e.g., CUDA, TBB, OpenMP).
• CUB: A lower-level, CUDA-specific library for "speed-of-light" parallel algorithms. It provides device-wide algorithms and cooperative algorithms (like block-wide reduction or warp-wide scan) for building custom kernels.
• libcudacxx: The CUDA C++ Standard Library. It provides a C++ Standard Library implementation that works on both host and device, along with abstractions for CUDA-specific hardware features like synchronization primitives, cache control, and atomics.

The mdspan extended API in CCCL provides specialized accessors, layout policies, and conversion utilities to handle various CUDA memory spaces and data formats.

Available features include:

• Host/Device/Managed Accessors: Support for different CUDA memory spaces.
• Layout Stride Relaxed: Layout mapping policies supporting negative/zero strides and offsets.
• Restrict Accessor: Support for the *restrict aliasing policy.
• Shared Memory Accessor: Specialized mdspan and accessor for CUDA shared memory.
• DLPack Interoperability: Conversion between mdspan and DLTensor formats.

The libcudacxx extended API provides mechanisms to represent, bind, and apply memory access properties to pointers. These properties allow for more explicit communication of memory access intent (such as read-only or read-write constraints) within CUDA code.

Key components include:

• cuda::access_property: The core abstraction representing a specific memory access property.
• cuda::annotated_ptr: A wrapper that binds an access property directly to a pointer.
• cuda::apply_access_property: A mechanism to apply an access property to memory.
• cuda::associate_access_property: A way to associate an access property with a raw pointer.

The libcudacxx extended math API provides specialized integer and high-performance mathematical functions for CUDA development. These functions are categorized by their operation, such as integer logarithms, power-of-two operations, and optimized division/modulo routines.

Note that availability of these functions depends on both the CCCL version and the CUDA Toolkit (CTK) version. For example, functions like sincos require CCCL 3.3.0 and CUDA 13.3 or later.

The cudax graphs library provides C++ wrappers to create and manage CUDA graphs. It uses owning and non-owning wrappers around standard CUDA graph types to provide a more ergonomic interface.

Warning: This library is currently under construction and is not yet ready for production use.

The <cuda/memory_resource> header provides a standard C++ interface for heterogeneous, stream-ordered memory allocation. It is designed specifically for CUDA C++ developers to handle memory across different device/host boundaries and utilize asynchronous stream-ordered allocation mechanisms.

Key characteristics:

• Heterogeneous Support: Unlike std::pmr::memory_resource, this interface is designed to verify and manage memory accessibility across different device types.
• Stream-Ordered: Built to support CUDA stream-ordered memory allocation.
• Relationship to RMM: This header defines the interface (cuda::mr). The RAPIDS Memory Manager (RMM) project provides the actual implementations of these interfaces.