rapids-singlecell

repository·main·Indexed 18 days ago

https://github.com/scverse/rapids-singlecell

A GPU-accelerated library for single-cell analysis featuring an AnnData-first API. It leverages CuPy and NVIDIA RAPIDS to provide high-performance implementations of common workflows, including preprocessing (pp), tools (tl), spatial analysis (gr), perturbation analysis (ptg), and biological activity extraction (dcg). Designed for compatibility with the Scanpy ecosystem, it allows efficient data movement between CPU and GPU and integrates functionalities from Squidpy, decoupler, and pertpy.

Tokens
19.7K
Snippets
58
Records
99
Agent score
63%

What's inside rapids-singlecell

  1. Overview of rapids-singlecell

    main

    rapids-singlecell is a library designed for GPU-accelerated single-cell analysis. It follows an AnnData-first API, making it highly compatible with the Scanpy ecosystem. It integrates selected functionalities from other scverse tools like Squidpy (spatial analysis), decoupler (deconvolution/enrichment), and pertpy (perturbation analysis).

    Key features include:

    • GPU Acceleration: Common single-cell workflows are executed on the GPU using CuPy and NVIDIA RAPIDS to handle large datasets efficiently.
    • Ecosystem Compatibility: Designed to work seamlessly with Scanpy APIs and AnnData objects.
  2. Use GPU-accelerated perturbation analysis with `rapids_singlecell.ptg`

    main
    The rapids_singlecell.ptg module provides GPU-accelerated implementations of methods from the pertpy library for perturbation analysis. It includes accelerated versions of Distance-based methods, GuideAssignment, Mixscape, and Mixscale.
  3. Use decoupler-GPU (`dcg`) for accelerated biological activity extraction

    main

    The rapids_singlecell.dcg module (decoupler-GPU or dcg) provides GPU-accelerated implementations of statistical methods originally found in decoupler.mt. These methods are used to extract biological activities from single-cell data.

    Available accelerated methods include:

    • dcg.mlm (Multi-layer Model)
    • dcg.ulm (Univariate Linear Model)
    • dcg.aucell (AUC-based method)
    • dcg.waggr (Waggr method)
    • dcg.zscore (Z-score based method)
  4. Project structure of rapids-singlecell

    main

    The repository is organized as follows:

    • src/rapids_singlecell/: Python source code.
      • preprocessing/: pp module (normalization, scaling, etc.).
      • tools/: tl module (PCA, UMAP, clustering, etc.).
      • squidpy_gpu/: Spatial analysis utilities.
      • pertpy_gpu/: Perturbation analysis utilities.
      • decoupler_gpu/: Pathway analysis utilities.
      • get/: CPU/GPU data transfer utilities.
      • _cuda/: Compiled CUDA kernels (nanobind).
        • nb_types.h: Shared ndarray type aliases.
        • <module>/: Individual kernel modules containing .cu bindings and .cuh implementations.
    • tests/: pytest test suite.
    • docs/: Sphinx documentation.
    • CMakeLists.txt: Build configuration for CUDA extensions.
  5. Kernel implementation conventions

    main

    When writing nanobind CUDA extensions, follow these conventions:

    • Launch Wrappers: Each kernel launch wrapper should be a static inline function in the .cu file.
    • Arguments: Use nb::kw_only() to separate data arguments from configuration arguments.
    • Streams: Accept std::uintptr_t stream as the last parameter (default 0) to support stream-based execution.
    • File Separation: Keep kernel logic in .cuh headers and bindings in .cu files.
    • Lazy Loading: Always import modules via rapids_singlecell._cuda. The package handles ImportError automatically; if the extension is unavailable (e.g., during docs builds), the import returns None instead of crashing.

    Example Import Pattern:

    from rapids_singlecell._cuda import _my_module_cuda as _my
    
    def my_function(adata):
        # _my is either the real module or None
        if _my is not None:
            _my.kernel(...)
    from rapids_singlecell._cuda import _my_module_cuda as _my
    
    def my_function(adata):
        # _my is either the real module or None
        _my.kernel(...)
  6. AnnData .raw uses CPU memory

    main

    Even if you move your main data to the GPU using rsc.get.anndata_to_GPU(adata), the .raw attribute in an AnnData object remains in CPU memory (NumPy/SciPy).

    This is critical because RSC functions using use_raw=True will receive CPU-resident data, which may trigger different execution paths (e.g., single-GPU instead of multi-GPU) compared to GPU-resident data.

    rsc.get.anndata_to_GPU(adata)
    adata.raw = adata.copy()
    
    print(type(adata.X))      # CuPy or cupyx: GPU
    print(type(adata.raw.X))  # NumPy or SciPy: CPU
  7. How memory management works in rapids-singlecell

    main

    rapids-singlecell integrates with the RAPIDS Memory Manager (rmm) to handle large-scale datasets efficiently. Upon importing rapids-singlecell, rmm is automatically invoked to modify the default allocator for cupy.

    While this integration typically results in minimal performance trade-offs, certain functions like ~.pp.harmony_integrate may experience more significant performance impacts.

    Users can manually configure or overwrite the default behavior using rmm.reinitialize. Crucial: You must configure RMM before creating any GPU arrays. Reinitializing while existing RMM allocations are still alive results in undefined behavior.

  8. How nanobind CUDA kernels work

    main

    GPU-accelerated functions are implemented as nanobind C++ extensions. Each module resides in src/rapids_singlecell/_cuda/<module>/ and consists of a .cu file (bindings and launch wrappers) and one or more .cuh headers (kernel implementations).

    Memory Layout and Type Aliases: Use the shared header nb_types.h to select the correct cuda_array alias based on your kernel's access pattern. Nanobind will reject arrays with incorrect layouts at runtime.

    • cuda_array<T>: No contiguity constraint.
    • cuda_array_c<T>: C-contiguous (row-major).
    • cuda_array_f<T>: F-contiguous (column-major). Use this for column-by-column indexing (e.g., data + col * n_rows).
    • cuda_array_contig<T, Contig>: Parameterized contiguity.
    // Example usage of type aliases from nb_types.h
    void my_kernel(cuda_array_f<float> data) {
        // Accessing data column-by-column
    }
  9. Run Jupyter notebooks on separate GPUs

    main

    To assign different GPUs to different Jupyter notebooks, set the CUDA_VISIBLE_DEVICES environment variable in the very first cell of a fresh kernel using the IPython %env magic command.

    Warning: Do not use !export CUDA_VISIBLE_DEVICES=... as shell commands run in child processes and will not affect the notebook kernel. You must restart the kernel if you need to change the assignment.

    # In Notebook A
    %env CUDA_VISIBLE_DEVICES=0
    
    # In Notebook B
    %env CUDA_VISIBLE_DEVICES=1
    
    # Verify in the next cell
    import cupy as cp
    assert cp.cuda.runtime.getDeviceCount() == 1
    assert cp.cuda.Device().id == 0
  10. Use Squidpy-compatible API via rsc.gr

    main

    GPU-accelerated implementations of common squidpy workflows are available in the rsc.gr module.

    rsc.gr.spatial_autocorr(
        adata,
        connectivity_key="spatial_connectivities",
        mode="moran",
        n_perms=500,
    )
    rsc.gr.co_occurrence(adata, cluster_key="labels", interval=50)
    rsc.gr.ligrec(adata, cluster_key="labels", n_perms=1000)
  11. Install rapids-singlecell from source

    main

    To develop on rapids-singlecell, clone the repository with submodules and install in editable mode. An editable install compiles the CUDA kernels for your local GPU architecture, placing compiled .so modules and .pyi type stubs in src/rapids_singlecell/_cuda/.

    Prerequisites:

    • NVIDIA GPU with CUDA support
    • A package manager: micromamba, conda/mamba, or uv
    • A RAPIDS environment (e.g., conda rapids-26.04 or pip-installed RAPIDS)
    • CUDA toolkit version constraint: For RAPIDS 26.04, you must use CUDA ≥ 12.9 or ≤ 12.5 when building from source to avoid a CCCL bug. If you are on CUDA 12.6–12.8, use the prebuilt wheel instead: pip install rapids-singlecell-cu12.
    git clone --recurse-submodules https://github.com/scverse/rapids-singlecell.git
    cd rapids-singlecell
    (uv) pip install -e ".[test]"