NVIDIA DALI Documentation

NVIDIA DALI (Data Loading Library) is a GPU-accelerated library designed to accelerate deep learning applications by offloading data loading and pre-processing (loading, decoding, cropping, resizing, etc.) from the CPU to the GPU.

Key features include:

• High Performance: Uses a custom execution engine with prefetching, parallel execution, and batch processing.
• Portability: Pipelines can be retargeted across TensorFlow, PyTorch, PaddlePaddle, and JAX.
• Format Support: Supports LMDB, RecordIO, TFRecord, COCO, JPEG, JPEG 2000, WAV, FLAC, OGG, H.264, VP9, and HEVC.
• Scalability: Supports CPU/GPU execution and scales across multiple GPUs.
• Advanced Integration: Supports GPUDirect Storage and NVIDIA Triton Inference Server via the DALI TRITON Backend.

DALI Dynamic is an experimental feature that introduces an imperative execution model with lazy evaluation to NVIDIA DALI. It is designed to complement the existing graph-based pipeline execution by enabling seamless integration into Python workflows, making it easier to debug and prototype pre-processing pipelines.

Key advantages include:

• Direct control of operator execution within iterative workflows.
• Simplified integration with existing Python code and libraries.
• Incremental writing and debugging of pipelines.
• Asynchronous operator evaluation for improved performance.

Note: DALI Dynamic does not replace the graph-based Pipeline API; it serves as an alternative interface for a better Python experience. You can transition between the two modes easily as they use the same operators.

Dynamic mode in nvidia.dali.experimental.dynamic is generally thread-safe and supports free-threaded Python. Operators can be called concurrently from multiple threads, and Tensor and Batch objects can be safely passed between threads.

Constraint: A single EvalContext instance must not be active in multiple threads simultaneously. While the default evaluation context is thread-local, manually created EvalContext objects shared across threads will cause exceptions.

DALI provides two distinct API modes:

• DALI Pipeline API: Uses a fixed graph of operations defined upfront. It is optimized for performance and uses the Pipeline class, nvidia.dali.fn operations, and nvidia.dali.math expressions.
• DALI Dynamic API: Allows running operators in an imperative manner without defining the data processing graph upfront. This offers more flexibility but incurs a small performance overhead.

DALI provides two ways to define operations: the recommended functional API (dali.fn) and the legacy operator object API (dali.ops).

• Functional API (dali.fn): Uses snake_case names. Operations are typically called directly within a pipeline context. This is the current recommended approach.
• Legacy Operator API (dali.ops): Uses camelCase names. This API separates the definition of an operator (where static arguments are set during instantiation) from its use in a pipeline.

Example mapping: dali.fn.crop_mirror_normalize is the functional counterpart to the legacy dali.ops.CropMirrorNormalize.

You can mix both APIs within a single pipeline.

The nvidia.dali.auto_aug library is structured around three core concepts:

• augmentation: An image processing operation. DALI provides common augmentations and allows you to implement new ones using the @augmentation decorator from nvidia.dali.auto_aug.core.augmentation.
• policy: A collection of augmentations and parameters (probability and strength) that define how to apply transformations to input images.
• apply operation: A function that invokes a specific policy on a batch of images within a DALI pipeline.

To provide AI agents with guidance on the DALI Dynamic Mode API and best practices, you can install the dali-dynamic-mode skill using npx skills.

npx skills add nvidia/skills --skill dali-dynamic-mode

Before installing DALI, ensure your environment meets the following requirements:

1. OS: Linux x64.
2. NVIDIA Driver: Must support CUDA 12.0 or later (e.g., driver version 525.60 or later).
3. CUDA Toolkit: Must be installed as it is linked dynamically.
4. Deep Learning Frameworks (Optional): PyTorch, TensorFlow, JAX, or PaddlePaddle.

DALI uses broadcasting to handle operations between tensors of different shapes.

Broadcasting Rules:

• A scalar is broadcast to all output values.
• If one operand has size 1 in a dimension, it is broadcast along that dimension to match the other operand.
• Shape Extension: If tensors have a different number of dimensions, DALI pads the shapes with outer unit dimensions (e.g., a shape (3,) is treated as (1, 1, 3) when operating with a (480, 640, 3) tensor).

Limitations: Broadcasting complexity is limited to a maximum of six alternating broadcast/non-broadcast groups.