cuPyNumeric Documentation

NVIDIA cuPyNumeric implements the NumPy API on top of the Legate framework. It provides transparent accelerated computing that scales across different hardware configurations, including:

• Single CPU
• Single GPU
• Multi-node, multi-GPU systems (e.g., scaling to 2048 A100 GPUs on a DGX SuperPOD).

Because it implements the NumPy API, existing NumPy-based code (such as Python CFD courses) can often be run completely unmodified to achieve high-performance scaling.

cuPyNumeric is a high-performance array computing library that implements the NumPy API on top of the Legate framework. It is designed to allow existing NumPy workflows to run on GPUs and distributed systems with minimal to no code changes.

Key Capabilities:

• Seamless Scaling: Scale workloads from a single CPU to a single GPU, and up to thousands of GPUs across multiple nodes.
• Use Cases: Large-scale data analysis, complex simulations, and machine learning.

A Legate task is a Python function used to extend cuPyNumeric with custom, scale-out algorithms. By annotating a function with the @legate.core.task.task decorator, you inform the Legate runtime that the function is a parallel task capable of being distributed across multiple CPUs, GPUs, and nodes.

Legate tasks are automatically available when using cuPyNumeric and require no additional installation. They allow you to implement novel research algorithms or leverage external libraries while maintaining transparent scaling across distributed resources.

import legate.core.task

@legate.core.task.task
def my_parallel_task(input_data, output_data):
    # Task logic here
    pass

cuPyNumeric uses a lazy / deferred, asynchronous, and task-parallel execution model. When you call a NumPy function (e.g., c = a + b), the Python call returns immediately with a DeferredArray thunk. No actual computation occurs at the moment of the call. Instead, a task is built and submitted to the Legate runtime, which schedules the work on available processors (GPU, OMP, or CPU) asynchronously.

Key Mental Model:

• Submission: Synchronous from Python's perspective (the API returns immediately).
• Execution: Asynchronous; the Legate runtime dispatches kernels/tasks.
• Completion: Invisible to Python until a sync point is reached.

In NumPy, functions like np.diag, np.flip, .flat, and .flatten() often return views of the original array. In cuPyNumeric, these operations return copies.

Warning: If you mutate the result of these functions expecting to modify the original array, your changes will be lost because you are mutating a copy.

Fix: If you need to mutate the data, write through to the original array using explicit indexing.

# BAD: Mutation fails because np.diag returns a copy in cuPyNumeric
d = np.diag(matrix)
d[0] = 5  # matrix remains unchanged

# GOOD: Explicitly write to the original
matrix[range(n), range(n)] = 5.0

cuPyNumeric only supports numeric datatypes. dtype=object arrays are not supported on the distributed path.

Fix: Restructure data into numeric representations:

• Variable-length strings: Use fixed-width or pad with sentinels + a lengths array.
• Heterogeneous records: Use a Structure-of-Arrays (one numeric array per field).
• Variable-length sequences: Use flat concatenation + an offsets array.

Using the out= parameter in NumPy functions allows you to reuse existing FBMEM allocations instead of creating new temporaries. This is critical for performance in hot loops.

Why it matters:

• cuPyNumeric does not JIT-fuse adjacent kernels in the mainline. Without out=, an expression like result = a + b * c allocates two intermediate temporary arrays.
• Avoiding temporaries reduces memory churn, prevents fragmentation, and reduces scheduling overhead.

Caveats:

• The out array must match the required shape and dtype of the operation.
• Some operations (like reductions with keepdims=False) may not accept out= if the output shape differs from the input.

np.add(a, b, out=result)
np.multiply(result, scale, out=result)
np.matmul(A, B, out=C)
np.sum(arr, axis=0, out=row_sums)

Functions like linalg.qr and linalg.svd are single-device only in cuPyNumeric; adding more GPUs will not speed up a single factorization.

To achieve multi-GPU scaling, you must batch your operations along the leading axis. This transforms the problem into a data-parallel workload where multiple independent factorizations are distributed across GPUs.