DOCSEARCHbyAlgolia
Submit Docs

bazel-lib

repository·main·Indexed 41 days ago

https://github.com/bazel-contrib/bazel-lib

A collection of base Starlark libraries and basic Bazel rules for constructing rulesets and BUILD files. It includes CLI utilities such as copy_directory, copy_to_directory, expand_template, and spawn_binary. Formerly known as aspect_bazel_lib prior to the 3.0 release.

Tokens
3.2K
Snippets
7
Records
14
Agent score
52%

What's inside bazel-lib

  1. Understand the relationship between bazel_lib and bazel-skylib

    main
    The bazel_lib module depends on bazel-skylib. While many of these utilities could theoretically be part of bazel-skylib, bazel-skylib has a narrow scope and is no longer accepting feature requests. There is a long-term possibility that bazel_lib may eventually replace bazel-skylib in an ABI-compatible manner.

  2. Install bazel_lib using WORKSPACE

    main

    To use a specific commit of bazel_lib in a WORKSPACE file, follow these steps:

    1. Replace the existing url (pointing to a release tarball) with a GitHub source archive URL using the commit SHA (e.g., https://github.com/bazel-contrib/bazel-lib/archive/abc123.tar.gz).
    2. Update strip_prefix to match the archive's prefix format (e.g., bazel-lib-abc123).
    3. Update the sha256 checksum. You can do this by commenting out the line and letting Bazel print the correct value during the next build.
    4. Load and call the Go dependency helper: load("@bazel_lib//:deps.bzl", "go_dependencies") followed by go_dependencies().

    Warning: GitHub source archives may not provide strong guarantees on sha256 stability.

  3. Transition from aspect_bazel_lib to bazel_lib (v3.0+)

    main

    The module has been renamed from aspect_bazel_lib (used in 2.x releases) to bazel_lib starting with the 3.0 release. This change follows Aspect's donation of the project to the Linux Foundation to allow broader community participation.

    Migration Notes:

    • The old aspect_bazel_lib and the new bazel_lib can be used in parallel during a transition period.
    • Certain utilities, specifically tar, jq, and yq, have been moved out of bazel_lib into their own dedicated modules as part of the 3.0 release.

  4. Install bazel_lib using Bzlmod

    main

    To use a specific commit of bazel_lib instead of a tagged release in your MODULE.bazel file, use the git_override directive.

    Note: Using a commit instead of a release requires building helper programs (like copy_to_directory, expand_template, and spawn_binary) from Go source, which increases your project's Go development dependencies.

    # Automatically picks up new Go dev dependencies
git_override(
    module_name = "bazel_lib",
    commit = "abc123",
    remote = "git@github.com:bazel-contrib/bazel-lib.git",
)

  5. Use spawn_binary to wrap POSIX processes

    main

    The spawn_binary CLI tool is a thin wrapper around arbitrary POSIX processes. It is primarily used by the run_binary rule to interpose on spawned programs, allowing for features like output redirection, exit code capture, and custom failure conditions while ensuring the child process behaves as if Bazel launched it directly.

    Key Design Principles:

    • Transparency: Environment variables and stdin are passed through verbatim. Streams are only redirected if explicitly requested via flags.
    • Signal Relay: Termination signals (like those from Bazel cancelling or timing out an action) are relayed to the child process to ensure the entire process tree is torn down.
    • Path Resolution: The tool resolves the command to an absolute path relative to the execution root. This ensures that even if --chdir is used, the tool correctly identifies the executable and doesn't accidentally perform a PATH lookup.
    spawn_binary [flags] -- <tool> [args...]

  6. Understand the path transformation pipeline

    main

    When copy_to_directory calculates the destination path for a file, it applies transformations in a specific, deterministic order. Understanding this order is critical for correctly configuring replace_prefixes and root_paths.

    Transformation Order

    1. include_external_repositories: Filters files based on their Bazel workspace name.
    2. include_srcs_packages: Filters files based on their Bazel package name.
    3. exclude_srcs_packages: Excludes files based on their Bazel package name.
    4. root_paths: Strips the longest matching prefix from the output path.
    5. include_srcs_patterns: Filters the resulting path against inclusion glob patterns.
    6. exclude_srcs_patterns: Filters the resulting path against exclusion glob patterns.
    7. replace_prefixes: Applies prefix replacements to the final path.

    Note: If a file is excluded at any step, it will not be copied.

  7. How spawn_binary handles action failures

    main

    The spawn_binary wrapper determines whether the overall Bazel action should fail based on the child process's exit code and the provided configuration. The logic follows this precedence:

    1. --fail-on: If this flag is set, the action fails only if the child's exit code matches one of the specified values. This takes precedence over other logic.
    2. --exit-code-out: If this flag is set (and --fail-on is not), the action is treated as a success regardless of the child's exit code. The exit code is simply written to the specified file.
    3. Default Behavior: If neither of the above is set, the action fails if the child process exits with a non-zero status.

    Note on Exit Status: If the child exits with code 0 but the action is considered a failure (e.g., because of a specific configuration not covered by the logic above), the wrapper will exit with status 1 to ensure Bazel detects the failure.

  8. Configure copy_to_directory via JSON

    main

    The behavior of the copy_to_directory tool is controlled by a JSON configuration file. The configuration defines which files to process, where they should go, and how their paths should be transformed.

    Configuration Schema

    KeyTypeDescription
    allow_overwritesbooleanIf true, allows multiple source files to map to the same destination path (the last one wins). If false, the tool errors on duplicates.
    dststringThe destination directory where files will be copied.
    exclude_srcs_packages[]stringA list of glob patterns for Bazel packages to exclude.
    exclude_srcs_patterns[]stringA list of glob patterns for file paths to exclude.
    files[]fileInfoAn array of fileInfo objects specifying the source files/directories to copy.
    include_external_repositories[]stringGlob patterns for external Bazel workspaces to include.
    include_srcs_packages[]stringA list of glob patterns for Bazel packages to include.
    include_srcs_patterns[]stringA list of glob patterns for file paths to include.
    replace_prefixesmap[string]stringA map of prefix transformations. If a destination path matches a key, the prefix is replaced by the corresponding value.
    root_paths[]stringA list of glob patterns used to strip prefixes from the output path.
    preserve_mtimebooleanIf true, attempts to preserve the original modification time of the files.
    verbosebooleanIf true, enables verbose logging during the copy process.

    fileInfo Object

    Each entry in the files array must be a fileInfo object:

    FieldTypeDescription
    packagestringThe Bazel package name.
    pathstringThe absolute or relative path to the source file/directory.
    root_pathstringThe root path of the file.
    short_pathstringThe path relative to the source root.
    workspacestringThe name of the workspace the file belongs to.
    workspace_pathstringThe path within the workspace.
    hardlinkbooleanWhether to use hardlinks for the copy.

  9. Reference: spawn_binary CLI flags

    main

    The following flags are supported by the spawn_binary CLI tool. Note that all flags must precede the -- separator.

    --stdout=FILE         capture the program's stdout into FILE
--stderr=FILE         capture the program's stderr into FILE
--exit-code-out=FILE  write the program's exit code into FILE
--fail-on=CODE       treat CODE as an action failure (repeatable; comma-separated
                     values in one flag are also accepted)
--chdir=DIR           run the program with DIR as its working directory
--silent-on-success   only forward non-captured stdout/stderr to the console
                      when the program exits non-zero

  10. Reference: expand_template CLI arguments

    main

    The expand_template command requires exactly 6 positional arguments:

    ArgumentDescription
    <template>The source template file to be processed.
    <out>The destination path for the expanded content.
    <substitutions_json>A JSON file mapping keys to replacement strings.
    <volatile_status_file>A status file for volatile variables.
    <stable_status_file>A status file for stable variables.
    <executable>Set to true or 1 to make the output file executable.
    Usage: expand_template <template> <out> <substitutions_json> <volatile_status_file> <stable_status_file> <executable>

  11. Use the expand_template CLI tool

    main

    The expand_template tool performs template expansion by replacing placeholders in a template file with values from a JSON substitution map and status files. It supports two-pass substitution: first, it resolves tokens in the substitution values using status files (using {{TOKEN}} syntax), and then it applies the final substitution map to the template content.

    Arguments:

    1. <template>: Path to the input template file.
    2. <out>: Path where the expanded file will be written.
    3. <substitutions_json>: Path to a JSON file containing a map of string: string substitutions.
    4. <volatile_status_file>: Path to a status file containing volatile key-value pairs.
    5. <stable_status_file>: Path to a status file containing stable key-value pairs.
    6. <executable>: A boolean flag (true or 1) to set the output file permissions to 0777 (executable) instead of 0666.

    Status File Format: Status files must contain lines in the format KEY VALUE. The tool uses a regex to capture the key and the remainder of the line as the value.

    # Example usage: expanding a template with substitutions and status files
./expand_template template.txt output.txt subs.json volatile.status stable.status true

  12. Use the copy_directory CLI tool

    main

    The copy_directory tool is a CLI utility used to copy a directory from a source path to a destination path. It supports concurrent copying using a worker pool and provides options for hardlinking, verbosity, and preserving modification times.

    Usage: copy_directory <src> <dst> [flags]

    Flags:

    • --hardlink: Use hardlinks instead of copying file contents.
    • --verbose: Enable verbose output during the copy process.
    • --preserve-mtime: Preserve the modification time (mtime) of the files.
    copy_directory src dst [--hardlink] [--verbose] [--preserve-mtime]