DOCSEARCHbyAlgolia
Submit Docs

dstack Documentation

repository·master·Indexed 19 days ago

https://github.com/dstack-tee/dstack

An open framework for confidential AI that enables the deployment of Docker containers inside Trusted Execution Environments (TEEs) such as Intel TDX and AMD SEV-SNP. dstack provides hardware-rooted security, cryptographic privacy guarantees, and native support for NVIDIA Confidential Computing GPUs. The documentation covers self-hosting tutorials, security models, VMM and Gateway deployment, and the Key Management Service (KMS) including the auth-eth-bun Ethereum backend.

Tokens
279K
Snippets
831
Records
1.2K
Agent score
67%

What's inside dstack

  1. Available dstack SDKs by language

    master

    dstack provides client libraries for interacting with the guest agent from within a TEE. The available SDKs are located in the following directories:

    • Python: sdk/python
    • JavaScript/TypeScript: sdk/js
    • Rust: sdk/rust
    • Go: sdk/go

  3. What is local-key-provider

    master

    The local-key-provider is dstack's SGX-backed bootstrap key provider designed for TDX guests. It is protocol- and cryptography-compatible with the previous gramine-sealing-key-provider.

    It facilitates secure key provisioning by performing the following steps for each request:

    1. Verifies the guest's TDX quote using DCAP.
    2. Ensures the SGX and TDX quotes share the same quoting-enclave ID.
    3. Derives a 32-byte key using SHA-256(SGX sealing key || MRTD || RTMR0..3).
    4. Encrypts the derived key using the libsodium sealed-box format and the X25519 public key found in the TDX report data.
    5. Returns the ciphertext along with an SGX quote that binds its SHA-256 digest.

  4. Overview of dstack-gateway architecture

    master

    dstack-gateway is a distributed reverse proxy gateway designed for dstack services. It provides TLS termination, SNI routing, and automatic certificate management via the ACME protocol (Let's Encrypt).

    In a cluster deployment, multiple gateway nodes automatically sync state to ensure high availability. When a Confidential Virtual Machine (CVM) starts, it registers with a gateway, and the cluster syncs the CVM's information (including its WireGuard public key) so all gateway nodes can establish secure WireGuard tunnels to that CVM.

  5. Overview of sca (Self-Contained App Builder)

    master

    sca is a tool that packages an application directly into an app-compose.json file. This allows a dstack Confidential Virtual Machine (CVM) to run the application without requiring Docker or a container registry.

    How it works:

    1. You define a rootfs/ directory tree that mirrors the desired CVM filesystem.
    2. sca build packs this tree into a deterministic tar.gz and encodes it as base64 within app-compose.json.
    3. At boot, a generated bash script extracts the tree and uses systemd to supervise your application (e.g., using Restart=always).

    Key Benefits:

    • No Docker/Registry: Ship static binaries or scripts inline.
    • Security: The entire app-compose.json (including the embedded rootfs) is hashed into a compose-hash and extended to RTMR3, ensuring the exact bytes you ship are covered by remote attestation and gated by an on-chain whitelist.

  6. Understand Intel TDX RTMR3 measurements

    master

    On Intel TDX, RTMR3 is an append-only runtime measurement register. dstack extends RTMR3 during guest boot with measurements for compose-hash, key-provider, instance-id, and other runtime events.

    If your application requires additional runtime measurements, you must extend RTMR3 using the native TDX or TSM measurement interfaces and manage the associated event-log for those extensions yourself.

  7. Understand the DstackKms and DstackApp Trust Model

    master

    The security of the dstack system relies on a distributed trust model between the KMS (Key Management Service) and individual Apps:

    • KMS Owner: Trusted for all KMS write operations, managing the OS-image whitelist, and upgrading the KMS implementation. They are not trusted for liveness (ownership can be transferred).
    • App Owner: Trusted for write operations on a specific App and upgrading that App's implementation. Each App owner is independent.
    • Pending Owner: Has the option to take ownership via acceptOwnership but has no privileges until that call is made.
    • Any EOA/Contract: Can call registerApp (permissionless) and read methods.
    • Registered IAppAuth contracts: Responsible for returning (bool, string) from isAppAllowed. Note that the KMS treats these outputs as untrusted; a malicious registered app could revert or consume gas, though it cannot mutate KMS state due to STATICCALL propagation.
    • Off-chain attestation pipeline: Trusted to emit the exact ASCII string "UpToDate" as tcbStatus when reporting a healthy TCB.

  8. Understand the Dstack GCP Attestation Flow (TDX + TPM)

    master

    Dstack uses a dual-layer attestation mechanism on GCP that combines Intel TDX quotes with TPM quotes to ensure the integrity of the confidential environment.

    The Flow

    1. TDX Layer: Generates a quote representing the hardware-protected Trusted Execution Environment (TEE).
    2. TPM Layer: Generates a quote that cryptographically binds the TDX quote to the guest's runtime state (PCRs).
    3. Verification: The verifier validates both the TDX collateral (via DCAP) and the TPM collateral, ensuring the TDX quote is part of the TPM's qualifying data and that the OS image identity matches the measured state.

    Key Components

    • TDX Quote Generation: tdx-attest::get_quote
    • TDX Event Log Reading: cc-eventlog::tdx::read_event_log
    • TPM Quote Generation: tpm-attest::TpmContext::create_quote
    • Verifier: dstack-attest (utilizing dcap-qvl and tpm-qvl)

  9. Important considerations for nerdctl-compose runner

    master

    When using the nerdctl-compose runner, be aware of the following constraints:

    • No Build Sections: The runner rejects Compose build sections. You must provide pre-built images. This ensures that lazy-pull images are converted before deployment and avoids requiring a BuildKit daemon inside the guest.
    • Feature Parity: While nerdctl compose implements common Docker Compose features, it is not a 1:1 replacement for all Docker-specific extensions. Test any applications using Docker socket mounts, custom runtimes, or advanced networking.
    • No Migration: Changing the runner (e.g., from docker-compose to nerdctl-compose) will recreate the application from scratch. It does not migrate existing containers between backends.

  10. How `os_image_hash` is verified and trusted

    master

    The os_image_hash in vm_config is verified by binding the hardware-signed quote to the software that actually booted. This prevents a host from substituting a malicious image.

    Verification Paths

    Full-image TDX Path

    1. The verifier obtains the OS image identified by os_image_hash.
    2. It checks the image checksum manifest.
    3. It recomputes the expected MRTD and RTMR0-2 from the image and VM configuration.
    4. It requires these recomputed values to match the measurements in the hardware quote.

    Unified Image Identity (TDX Lite, SEV-SNP, GCP TDX)

    For these paths, the os_image_hash is defined as sha256(sha256sum.txt). The sha256sum.txt is a manifest containing SHA-256 digests and relative filenames for all critical components (kernel, initrd, firmware, etc.).

    The verification process follows these steps:

    1. sha256(checksum_file) == os_image_hash.
    2. The checksum file must contain the expected measurement.*.cbor entry (e.g., measurement.tdx.cbor or measurement.snp.cbor), and that entry must hash to the supplied measurement material.
    3. The supplied measurement material must replay to the hardware-signed TDX MRTD/RTMR values, SEV-SNP launch MEASUREMENT/HOST_DATA, or the GCP TPM UKI event.

  11. Understand NVIDIA Attestation Proxy cache behavior

    master

    The proxy caches OCSP responses and NVIDIA RIM documents to ensure reliable attestation.

    Cache Lifecycle and Refreshing

    • Background Sweep: Every --refresh-interval (default: 10 minutes), the proxy attempts to renew entries that have consumed half their lifetime. This ensures a warm cache even during NVIDIA outages.
    • Synchronous Fallback: If an OCSP response has less than --ocsp-refresh-before (default: 5 minutes) validity remaining, the proxy performs an in-line refresh on the next request. Concurrent refreshes for the same entry are coalesced.
    • Expiration:
      • OCSP: Expired OCSP entries are never served. The proxy will not extend or serve an expired response.
      • RIM: Expired RIM documents may be served for up to --rim-max-stale (default: 7 days) if the upstream is unreachable. The guest verifies the signature regardless.

    TTL Defaults

    • OCSP: Up to --ocsp-max-ttl (default: 24 hours). If the response lacks a nextUpdate field, it uses one hour from thisUpdate.
    • RIM: Default 30-day TTL.

    Capacity

    Each cache kind is capped at 10,000 entries by default. Use --max-cache-entries-per-kind to adjust this limit. When the limit is reached, the oldest entry is evicted.

  12. Understand dstack Guest OS Image components

    master

    A dstack guest OS image is a collection of components required to boot a Confidential Virtual Machine (CVM) within a TDX-protected environment. These components are measured by TDX hardware during boot to create a cryptographic chain of trust.

    Core components include:

    • OVMF.fd: Virtual firmware (UEFI BIOS) that boots first and establishes TDX measurements.
    • bzImage: The Linux kernel compiled specifically for TDX guests.
    • initramfs.cpio.gz: The initial RAM filesystem containing early boot scripts.
    • rootfs.cpio: The root filesystem containing tappd and the container runtime.