DOCSEARCHbyAlgolia
Submit Docs

Please Build System

repository·master·Indexed 25 days ago

https://github.com/thought-machine/please

A high-performance, cross-language build system designed for extensibility and reproducibility. Please uses a hermetic caching model based on input hashes to enable fast, parallel builds. It features a hybrid build language (ASP), a consistent CLI via the `plz` command, and support for remote execution via the remote execution API and HTTP caching.

Tokens
17.5K
Snippets
60
Records
130
Agent score
82%

What's inside please

  1. Use built-in build rules in Please

    master
    Please provides a collection of built-in build rules to simplify common build tasks. These rules are organized by language and are written in the BUILD language. The core, low-level functions are located in builtins.build_defs, which contains functions known to the Please interpreter that may have custom, optimized implementations.

  2. Use HTTP Cache as a resource-based cache for Please

    master
    The http_cache tool implements a lightweight, resource-based HTTP server that please can use as a cache. It allows please to store files via PUT requests and retrieve them via GET requests. While any HTTP server (like nginx) can be used, http_cache is designed to be easy to configure for this specific purpose.

  3. Use the Please Alpine image for building

    master

    The Please Alpine image provides a minimal, canonical build environment for building Please and its associated tools in Linux. It is the environment used to release official binaries.

    Note: This image is optimized for building and is fairly minimal. It does not contain all the dependencies required to run all language-specific tests. If you need to run the full suite of language-specific tests, use the Ubuntu image instead.

  4. Core concepts of the Please build system

    master

    The Please build system is built around several central structures that manage the build lifecycle, graph, and configuration:

    • BuildTarget: Represents a specific task for the build system (created via build_rule(), filegroup(), or remote_file()).
    • BuildLabel: The unique identifier used to refer to build targets (e.g., //third_party/go:testify).
    • BuildGraph: A collection of all parsed build targets and subrepos.
    • Configuration: The combined configuration derived from .plzconfig files.
    • BuildState: Manages the current build state, including the build graph, configuration, and progress. It tracks whether the system should queue targets for testing or parsing. BuildState is cloned for subrepos, though parts like the build graph are shared.
    • Subrepo: A subrepo added to the build graph, created via subrepo() calls, specific architectures (e.g., ///linux_amd64//...), or the --arch command-line flag.

  5. Understand ASP syntax and limitations

    master

    ASP is a custom parser for BUILD files that uses a syntactically restricted subset of Python. When writing BUILD files using ASP, be aware of the following language constraints:

    Prohibited Keywords

    The following keywords are unavailable and cannot be used as identifiers:

    • import, try, except, finally, class, global, nonlocal, while, async

    Supported Types and Builtins

    • Supported Types: bool, int, str, list, dict, and functions.
    • Unsupported Types: float, complex, set, frozenset, bytes.
    • Dictionaries: Can only be keyed by strings. They iterate in a consistent order.
    • Builtins: Most Python builtin functions are unavailable.

    Supported Operators and Expressions

    • Operators: +, <, >, %, and, or, in, not in, is, is not, ==, >=, <=, !=.
    • Assignment: Standard = and the augmented assignment operator += are supported. Other augmented assignments (like -=) are not available.
    • Comprehensions: List and dict comprehensions are supported (up to two for clauses), but generator expressions are not.
    • String Interpolation: Supported via % and f-strings. Note that .format() is available but has an incomplete implementation and its use is discouraged.

    Features and Behavior

    • Assertions: The assert statement is supported, but errors cannot be caught because try/except are unavailable.
    • Type Annotations: Supported and checked at runtime. You can combine types using the | operator (e.g., foo:list|dict=[]).
    • Function Arguments: Evaluated at call time, making it safe to use mutable defaults like [] or {}.
    • List Mutability: ASP lists are more immutable than Python lists. While append and extend may work, they do not always follow standard Python semantics (e.g., they might not modify the original object if it originated from a surrounding scope).

    Best Practice: Instead of using append or extend, use the += augmented assignment operator to modify lists.

  6. Understand the package parse workflow

    master

    The parse package is responsible for parsing and interpreting BUILD files to populate the build graph. It uses the asp dialect (a Python dialect) for interpretation.

    The parsing lifecycle follows these steps:

    1. Synchronize on package parsing: Call state.SyncParsePackage(label).
      • If it returns an existing package, use it (the call blocks if parsing is already in flight).
      • If it returns nil, you are the first to parse this package and must proceed to parse it.
    2. Handle subrepos: If a subrepo label is present, the subrepo package must be parsed first. This requires waiting for the subrepo target that defines the package to be built.
    3. Populate the build graph: Parse the package and add it to the build graph. Once complete, mark the package as parsed to unblock other calls to state.SyncParsePackage(label).
    4. Trigger builds: If a specific target was queued for building, activate the target and re-queue it to trigger the build process.

  7. Use the alternative Ubuntu image for testing

    master

    The alternative Ubuntu image is a modified version of the standard Ubuntu image used specifically for testing compatibility in environments with different OS and package versions.

    Warning: This image is not recommended for normal production or development use.

    Notable differences from the standard image:

    • Uses Jammy instead of Noble.
    • Uses Go 1.23 (the standard image uses Go 1.24).

  8. Use the remote package for remote execution

    master

    The remote package provides a client for building please targets using the remote execution API.

    When building with remote execution, the client follows this workflow:

    1. Build the command and action protos from the Target.
    2. Check for a cached action result in the target metadata file.
    3. Check for a cached action result in the remote action cache.
    4. If no cache hit occurs, prepare the input directory and upload missing required files (e.g., srcs). Note that dependencies (deps) should already exist in the Content Addressable Storage (CAS) from previous builds.
    5. Submit the action for building and save the result to both the target metadata file and the local cache.

    Important Notes:

    • Filegroups: Since filegroups do not have a native action, the client generates a pseudo-action and action result locally to upload to the action cache.
    • Remote Files: These are built using the remote asset API.

  10. Create Kubernetes deployments with templated manifests

    master

    The k8s plugin allows you to create Kubernetes manifests that are automatically templated with the correct image tags from your docker_image rules.

    1. Install the plugin:

    plz init plugin k8s

    2. Define k8s_config in a BUILD file:

    k8s_config(
    name = "k8s",
    srcs = [
        "deployment.yaml",
        "service.yaml",
    ],
    containers = [":image"],
)

    3. Build and deploy: Building the k8s_config target generates templated YAML files in plz-out/gen/. The plugin also provides a _push target to apply these manifests to your cluster via kubectl:

    plz build //hello_service/k8s:k8s
plz run //hello_service/k8s:k8s_push

  11. Build Docker images with the Docker plugin

    master

    To build Docker images, first install the docker and shell plugins:

    plz init plugin shell && plz init plugin docker

    Use the docker_image rule in a BUILD file to define your image. Building the rule generates a shell script that performs the actual docker build command.

    Example docker_image rule:

    docker_image(
    name = "image",
    srcs = ["//hello_service"],
    dockerfile = "Dockerfile",
    base_image = "//common/docker:base",
)

    To build and run the image build script:

    plz build //hello_service/k8s:image
plz run //hello_service/k8s:image.sh