NVIDIA Container Toolkit

The nvidia-cdi-hook CLI is a tool designed to be called by a container runtime as part of the Container Device Interface (CDI) specification. It provides runtime hook capabilities to prepare the container environment (files, permissions, symlinks, etc.) before a container starts.

Note that nvidia-cdi-hook is not typically invoked manually by users to manage devices; instead, it is invoked by the container runtime based on instructions found in a CDI specification file. The CDI specification file itself is generated using the nvidia-ctk cdi generate command.

If the runtime is not in legacy mode, the docker --gpus flag will fail. To use the NVIDIA Container Runtime with Docker, you have two options:

1. Explicitly specify the runtime in your command:

   docker run --rm --gpus all --runtime=nvidia ubuntu:18.04

2. Set it as the default runtime by modifying /etc/docker/daemon.json:

   {
       "default-runtime": "nvidia",
       "runtimes": {
           "nvidia": {
               "path": "nvidia-container-runtime",
               "runtimeArgs": []
           }
       }
   }

The NVIDIA Container Toolkit can generate Container Device Interface (CDI) specifications to make NVIDIA devices accessible in vendor-agnostic container environments (like podman, containerd, or cri-o).

To generate the specification, use nvidia-ctk cdi generate. You can specify a destination file using the --output flag. To write to system directories like /etc/cdi/, you must use sudo.

The generated specification includes the following device names:

• nvidia.com/gpu=gpu{INDEX}: For non-MIG-enabled full GPUs.
• nvidia.com/gpu=mig{GPU_INDEX}:{MIG_INDEX}: For MIG-enabled devices.
• nvidia.com/gpu=all: Represents all available NVIDIA devices.

To enable NVIDIA device support via CDI, you must first generate a CDI specification file (in YAML format) for your NVIDIA-capable devices. This is done using the nvidia-ctk tool. The resulting YAML file contains the instructions that the container runtime will use to set up devices and execute hooks like nvidia-cdi-hook.

nvidia-ctk cdi generate

Use the scripts/release-packages.sh utility to automate the release process. This script pulls package images from the staging registry based on a git SHA, copies them to the package repository, and signs them with GPG keys.

Arguments:

• REPO: Either stable or experimental.
• PACKAGE_REPO_ROOT: The local path to the libnvidia-container repository (checked out to the gh-pages branch).
• REFERENCE (optional): The git SHA to be released. Defaults to HEAD if not provided.

Required Environment Variables:

• GPG_LOCAL_USER: The GPG user ID.
• MASTER_KEY_PATH: Path to the GPG master key.
• SUB_KEY_PATH: Path to the GPG subkey.

GPG_LOCAL_USER="GPG_USER" \
MASTER_KEY_PATH=/path/to/gpg-master.key \
SUB_KEY_PATH=/path/to/gpg-subkey.key \
    ./scripts/release-packages.sh REPO PACKAGE_REPO_ROOT [REFERENCE]

To build the components of the NVIDIA container stack, use the build-packages.sh script. You can specify a specific TARGET to build a particular platform, or omit it to build all valid release targets. Generated packages are placed in the dist folder.

Supported targets include:

• ubuntu18.04-amd64
• centos7-x86_64

./scripts/build-packages.sh TARGET

Before installing or using the NVIDIA Container Toolkit, ensure the following requirement is met:

• NVIDIA Driver: You must have the NVIDIA driver installed on your Linux distribution.

Note: You do not need to install the CUDA Toolkit on the host system; only the NVIDIA driver is required on the host.

When testing local changes, you can override the default locations of the individual components by setting the following environment variables to point to your local directories:

• LIBNVIDIA_CONTAINER_ROOT
• NVIDIA_CONTAINER_TOOLKIT_ROOT
• NVIDIA_CONTAINER_RUNTIME_ROOT
• NVIDIA_DOCKER_ROOT

Use the containerd setup command to configure the nvidia-container-runtime as a containerd runtime class. You must provide the path to the toolkit (typically /run/nvidia/toolkit).

If the --runtime-class flag is not specified, the runtime class will be named nvidia. Use --set-as-default to make the specified runtime class the default.

containerd setup \
    --runtime-class NAME \
        /run/nvidia/toolkit

Use the docker setup command to configure the nvidia-container-runtime as a Docker runtime. You must provide the path to the toolkit (typically /run/nvidia/toolkit).

By default, the toolkit is installed with the name nvidia and is set as the default Docker runtime. You can customize the runtime name using --runtime-name and control whether it becomes the default using --set-as-default.

docker setup \
    --runtime-name NAME \
        /run/nvidia/toolkit