DOCSEARCHbyAlgolia
Submit Docs

heph Build System

website·Indexed 30 days ago

https://hephbuild.github.io/heph

A build system featuring target-based caching, approval gates for irreversible actions, and a credential system for managing identities. It supports codegen modes (copy and in_place), remote shared caching, and deferred values for dynamic target options.

Tokens
58K
Snippets
317
Records
463
Agent score
72%

What's inside heph

  1. Overview of heph build system

    heph is an open-source build system and task orchestrator. It operates by defining targets, hashing their inputs, and executing actions in parallel within a sandbox. The system uses content-addressing for outputs to ensure that identical inputs produce byte-identical artifacts across different environments, such as local laptops and CI pipelines.

  2. Understand heph execution and caching enforcement

    heph ensures build correctness and caching guarantees through a three-step enforcement process:

    1. Input Hashing: Before execution, all sources, tool versions, environment variables, and rule definitions are hashed into a single digest.
    2. Hermetic Execution: Actions run within a sandbox that exposes only the explicitly declared inputs, preventing undeclared dependencies from affecting the build.
    3. Content-Addressed Outputs: The input digest serves as the key, and the resulting output bytes serve as the value, enabling reliable caching.

    Caveat: Targets that are non-hermetic opt out of these caching guarantees and should be used sparingly.

  3. Understand the heph build model and caching

    heph treats a build as a directed graph of targets. Each target defines its inputs and the action required to produce its outputs. To optimize performance, heph hashes all declared inputs (including sources, dependencies, and the toolchain). If the resulting digest matches a previous run, heph records a cache hit and returns the cached output instead of re-executing the build action. Any input not explicitly declared is invisible to the build sandbox.

  5. Use the Buildfile plugin to define targets via Starlark

    The buildfile plugin is a package provider that scans the workspace for BUILD files containing Starlark code. It evaluates these files to extract target definitions, including their drivers, configuration, labels, and transitive dependencies, which are then processed by the build engine. The plugin supports customizable BUILD file naming via glob patterns and allows the use of load() statements to share symbols and reuse common definitions across multiple BUILD files in the workspace.

  6. Understand the role of Drivers in the Go build plugin

    In this build system, a driver serves as the execution backend responsible for running specific types of targets. The plugin registers nine internal drivers: go_toolchain, go_golist, go_compile, go_testmain, go_lint, go_lint_gate, go_lint_fix, go_format, and go_format_check. These drivers are internal implementation details and should not be interacted with directly by the user.

  7. Run shell commands in sandboxed builds using pluginexec

    The pluginexec plugin manages the execution of shell commands within sandboxed target builds. It handles dependency injection, environment variable setup, and PATH resolution for required tools. It supports two execution drivers: exec and bash. Additionally, it enables interactive shell debugging through PTY allocation.

  8. Define Go build variants using provider_state

    Go targets in heph require a build variant to compile. A build variant is a named set of factors including target OS, architecture, build tags, and compiler/linker flags. Variants are declared using provider_state and are scoped to the Go module (the directory containing go.mod).

    Inheritance is supported via the inherit field. When inheriting, list fields (tags, goexperiment, gcflags, ldflags) are replaced by the child variant, not merged.

    provider_state(provider = "go", variants = {
    "base": {"goos": "linux", "goarch": "amd64"},
    "release": {
        "inherit": "base",
        "goexperiment": ["arenas"],
        "gcflags": ["-l"],
        "ldflags": ["-s", "-w"],
        "tags": ["prod"],
    },
})

  9. Use the heph Go provider for build target generation

    The go provider in heph provides Go language support by analyzing Go packages to automatically generate targets for building libraries, binaries, and tests. It handles package metadata reading, source file resolution, and module dependency management. The provider relies on several managed drivers to execute the underlying toolchain operations: go_toolchain for provisioning, go_golist for metadata analysis, go_compile for compilation, go_testmain for test main generation, and a suite of lint/format drivers including go_lint, go_lint_gate, go_lint_fix, go_format, and go_format_check.

  11. Configure tool cache access modes in heph

    The access setting in heph is an assertion about a tool's concurrency safety regarding its cache. Choosing the wrong mode can lead to cache corruption or unnecessary serialization of tasks.

    • exclusive (Default): Ensures only one consumer runs at a time by holding a lock for the entire duration of the run. Use this for most tools that do not explicitly document concurrent writer safety.
    • shared: Allows concurrent access. Use this only for tools that are safe under concurrent writers (e.g., Go's build and module caches, which are content-addressed).