BuildKit Documentation

The scheduler is a single-threaded, non-blocking event loop responsible for invoking operations to solve the build graph. It operates by solving edges rather than vertices.

Key concepts:

• Pipes: Interaction with the scheduler occurs via "pipes" between a sender and a receiver. One or both sides can be an edge instance.
• Unparking: When an edge receives an event via a pipe, the scheduler "unparks" it. The unpark handler must be non-blocking and execute quickly.
• Requests: The scheduler separates pipes into incoming requests (e.g., requests to retrieve a result or cache key) and outgoing requests (e.g., requests for async functions or requesting an input edge to reach a specific state).

Critical Rule for Implementation: To prevent deadlocks and panics, the unpark method must ensure that if it finishes without completing all incoming requests, it must have created outgoing requests. Conversely, if an incoming request remains pending, at least one outgoing request must exist.

BuildKit uses a specific set of terms to describe its architecture and build processes. Familiarize yourself with these concepts before contributing or integrating:

• LLB (Low-Level Build definition): A binary intermediate format used to define the dependency graph for build processes.
• Definition: The LLB serialized using protocol buffers, used for transport over gRPC interfaces.
• Frontend: Components that build LLB and issue requests to BuildKit's gRPC server (e.g., dockerfile.v0, gateway.v0).
• State: A helper object used by frontends to build LLBs from high-level concepts like images, shell executions, or mounts.
• Solver: An abstract interface that solves a graph of vertices and edges to find a final result.
• Vertex: A node in a build graph defining a content-addressable operation and its inputs.
• Op (Operation): Defines how a solver evaluates a vertex; executed in the worker (e.g., image sources, git sources, exec processes).
• Edge: A connection point between vertices that references a specific output from a vertex's operation.
• Result: The abstract interface return value of a solve, typically representing a container snapshot.
• Worker: A backend that runs OCI images (e.g., using runc or containerd).

The ARG instruction defines variables that can be passed to the builder at build time using the --build-arg <varname>=<value> flag. These variables can be used in instructions like FROM, ENV, WORKDIR, and RUN using ${VAR} or $VAR syntax.

Key Characteristics:

• Scope: An ARG variable is available from the line it is declared onwards. In multi-stage builds, an ARG declared in one stage is not automatically available in others unless redefined or inherited from a shared base stage.
• Persistence: Unlike ENV, ARG variables are not embedded in the final image and are not available in the running container.
• Security Warning: Do not use ARG for secrets (credentials, API tokens). They are visible in docker history and provenance attestations. Use RUN --mount=type=secret for secure secret handling.

ARG <name>[=<default value>] [<name>[=<default value>]...]

# Example with default values
FROM busybox
ARG user1=someuser
ARG buildno=1

To install BuildKit on Windows, you must run a PowerShell terminal as an Administrator.

1. Enable Windows Features: Enable Containers and Microsoft-Hyper-V.
2. Install containerd: Follow the official containerd installation guide for Windows. BuildKit currently only supports the containerd worker.
3. Configure CNI: Set up CNI networking (see CNI/Networking Setup section).
4. Download and Extract BuildKit: Download the latest release binaries and extract them.
5. Install Binaries: Move buildkitd.exe and buildctl.exe to a permanent directory (e.g., $Env:ProgramFiles\buildkit) and add that directory to your system PATH.

# 1. Enable Windows Features
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V, Containers -All

# 4. Download and extract
$url = "https://api.github.com/repos/moby/buildkit/releases/latest"
$version = (Invoke-RestMethod -Uri $url -UseBasicParsing).tag_name
$arch = "amd64"
curl.exe -fSLO https://github.com/moby/buildkit/releases/download/$version/buildkit-$version.windows-$arch.tar.gz
mv bin bin2
tar.exe xvf .\buildkit-$version.windows-$arch.tar.gz

# 5. Setup binaries and PATH
Copy-Item -Path ".\\bin" -Destination "$Env:ProgramFiles\\buildkit" -Recurse -Force
$Path = [Environment]::GetEnvironmentVariable("PATH", "Machine") + `
    [IO.Path]::PathSeparator + "$Env:ProgramFiles\\buildkit"
[Environment]::SetEnvironmentVariable( "Path", $Path, "Machine")
$Env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + `
    [System.Environment]::GetEnvironmentVariable("Path","User")