DOCSEARCHbyAlgolia
Submit Docs

RunPod Containers

repository·main·Indexed 42 days ago

https://github.com/runpod/containers

Official Dockerfiles and templates for RunPod, including Base images, AutoResearch environments, ComfyUI, NVIDIA PyTorch, and specialized PyTorch Cluster images for multi-node GPU clusters with RDMA/InfiniBand and Grafana monitoring.

Tokens
9.3K
Snippets
28
Records
50
Agent score
53%

What's inside RunPod Containers

  2. Overview of Runpod Base images

    main

    Runpod Base images are lean, flexible environments designed for machine learning workflows, supporting both CPU-only and GPU workloads. They can be used as standalone workspaces or as a foundation for building custom images.

    Key Features:

    • Python Support: Preinstalled versions 3.9–3.13 (3.10 is the default).
    • ML Ready: Includes essential libraries for scientific computing, computer vision, and machine learning, plus SLURM support.
    • Remote Access: Preconfigured SSH server for remote development and debugging.
    • Optimized Workspace: Optimized directory structures and package caches for faster dependency installation and execution.
    • Jupyter Support: Includes Notebook and JupyterLab with widgets/extensions.

  3. Overview of Runpod PyTorch optimized images

    main

    Runpod PyTorch provides pre-configured Docker images optimized for deep learning workflows. These containers are designed to eliminate compatibility guesswork by providing tested combinations of PyTorch and CUDA.

    Key features include:

    • Zero setup: PyTorch is ready to import immediately.
    • GPU acceleration: Full CUDA support is enabled out of the box.
    • Audio/Video support: torchaudio is matched to each PyTorch version. For images with PyTorch 2.9 and later, torchcodec is included, providing torchaudio.load and save wrappers and enabling GPU video decoding via NVDEC (replacing the removed torchaudio.io).

  4. Use the PyTorch Cluster image for multi-node GPU clusters

    main

    The pytorch-cluster image is a specialized variant of the standard runpod/pytorch image designed for multi-node GPU clusters. It adds two critical layers:

    1. RDMA / InfiniBand user-space stack: Includes libibverbs1, ibverbs-providers, rdma-core, ibverbs-utils, infiniband-diags, and perftest for high-speed interconnects.
    2. Monitoring Stack: A self-contained stack managed by supervisord that provides cluster-wide observability.

    Image Tagging Convention: Tags follow the standard PyTorch scheme with a -cluster suffix. Example: runpod/pytorch:<version>-cu1281-torch280-ubuntu2404-cluster.

    runpod/pytorch:<version>-cu<cuda_version>-torch<torch_version>-ubuntu<os_version>-cluster

  5. How the PyTorch Cluster monitoring stack works

    main

    The monitoring architecture is split between all nodes and a single head node (node-0) to ensure efficient data collection without excessive overhead.

    Node Roles

    • Every Node: Runs node_exporter (port :9100) and dcgm-exporter (port :9400) via supervisord to collect system and GPU metrics.
    • node-0 (Head Node): Runs the central monitoring services: Prometheus (127.0.0.1:9090) and Grafana (accessible via public port :8889 through an auth proxy).

    Startup Lifecycle

    The /post_start.sh hook executes at startup:

    1. On node-0: It scans /etc/hosts for peers starting with node- to configure Prometheus scrape targets for the entire cluster. It then starts Prometheus and Grafana.
    2. On all nodes: It starts supervisord to manage the exporters.

    Inspecting Services

    You can check the status of running monitoring services using supervisorctl status. Logs are located at /var/log/supervisor/.

    supervisorctl status

  6. Container Requirements for Runpod Platform Functionality

    main

    To ensure full compatibility with the Runpod platform, all container images must include the following dependencies:

    • nginx: Necessary for proxying ports to the user.
    • openssh-server: Required for providing SSH access to the container.
    • jupyterlab: Required for providing JupyterLab access to the container.

  7. Getting started with Runpod AutoResearch

    main

    Runpod AutoResearch provides a pre-configured environment for running autonomous ML research experiments. To use it, launch a pod using the AutoResearch template, connect your preferred AI coding agent (such as Claude Code or Cursor) via SSH, and initiate the experiment loop by instructing the agent to read the program specification.

    Workflow:

    1. Launch a pod with the AutoResearch template.
    2. Connect your coding agent via SSH.
    3. Command the agent to start the experiment loop using the following instruction: Read /workspace/autoresearch/program.md and let's kick off a new experiment!
    Read /workspace/autoresearch/program.md and let's kick off a new experiment!

  8. Verify NCCL connectivity and performance

    main

    To verify that NCCL (NVIDIA Collective Communications Library) is correctly configured and functioning within a RunPod container, run the provided verification script. This is useful for ensuring multi-GPU communication is working as expected.

    1. SSH into your running pod.
    2. Execute the check_nccl.sh script from the root directory of the container.
    ./check_nccl.sh

  9. Access Grafana dashboards via auto-login

    main

    Grafana is accessible on port 8889 (ensure this port is exposed on the node-0 pod). Access is managed via a seamless auto-login proxy that mirrors the Jupyter shared-token model.

    Authentication via JUPYTER_PASSWORD

    Instead of a separate login, Grafana uses the existing $JUPYTER_PASSWORD environment variable.

    1. Setup: Ensure the pod has the JUPYTER_PASSWORD environment variable set.
    2. Access: The console will open a URL in the format: https://<pod>-8889.<proxy-base>/?token=<JUPYTER_PASSWORD>.
    3. Mechanism: The proxy (config/grafana-proxy/auth_proxy.py) validates the ?token= parameter against JUPYTER_PASSWORD. If it matches, it proxies the request to the internal Grafana instance (:3300) and injects the X-WEBAUTH-User header.

    Configuration Notes

    • Custom Token: Set GRAFANA_PASSWORD if you want a specific token for Grafana that differs from Jupyter.
    • Dev Environments: If working in a dev environment, set RUNPOD_PROXY_BASE=dev-proxy.runpod.net so the startup script builds the correct root_url.
    • Security: Standard username/password login and HTTP basic auth are disabled. The only way to access Grafana is through the auth proxy.
    https://<pod>-8889.<proxy-base>/?token=<JUPYTER_PASSWORD>

  10. Update a Runpod template README via update-template-readme.sh

    main

    Use the update-template-readme.sh script to push updates to a Runpod template's README content using the Runpod API.

    Requirements:

    • The RUNPOD_API_KEY environment variable must be set with a valid API key.
    • Node.js must be installed (used for JSON escaping).
    • A README.md file must exist within the provided template path.

    Parameters:

    • template_id: The unique Runpod template ID to update.
    • template_path: The local filesystem path to the template directory containing the README.md file.
    # Update the PyTorch template README
./update-template-readme.sh abc123 official-templates/pytorch

# Update the ComfyUI template README
./update-template-readme.sh def456 official-templates/stable-diffusion-comfyui

  12. Verify Runpod container image signatures

    main

    Runpod uses keyless signing via Cosign and Sigstore. To ensure the integrity of a container image, you must verify its signature against the GitHub OIDC identity used during the build process.

    Prerequisites:

    • cosign version 2.0 or higher.
    • A tool to resolve image tags to digests (e.g., docker buildx or crane).

    Verification Workflow:

    1. Resolve the tag to a digest: Always verify the immutable sha256 digest rather than a mutable tag.
    2. Run cosign verify: Use the --certificate-identity-regexp to match the GitHub workflow URI and --certificate-oidc-issuer to specify the GitHub OIDC provider.

    Standard Verification Command:

    IMAGE="docker.io/runpod/base:1.0.7-ubuntu2204"

# 1. Resolve the tag to its immutable digest
DIGEST=$(docker buildx imagetools inspect "${IMAGE}" | awk '/^Digest:/ {print $2; exit}')

# 2. Verify the signature
cosign verify \
  --certificate-identity-regexp "^https://github.com/runpod/containers/\.github/workflows/.+" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  "docker.io/runpod/base@${DIGEST}"
    IMAGE="docker.io/runpod/base:1.0.7-ubuntu2204"

# 1. Resolve the tag to its immutable digest.
DIGEST=$(docker buildx imagetools inspect "${IMAGE}" \
           | awk '/^Digest:/ {print $2; exit}')
echo "Digest: ${DIGEST}"

# 2. Verify the signature against the GitHub OIDC identity that signed it.
cosign verify \
  --certificate-identity-regexp "^https://github.com/runpod/containers/\.github/workflows/.+" \
  --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
  "docker.io/runpod/base@${DIGEST}"