rules_nixpkgs

repository·master·Indexed 18 days ago

https://github.com/tweag/rules_nixpkgs

Bazel rules for hermetically importing Nix and Nixpkgs dependencies into Bazel builds. It provides mechanisms to import Nixpkgs via Git, local expressions, or Bzlmod module extensions, and allows exposing Nixpkgs packages and toolchains as Bazel targets. The library includes specialized modules like rules_nixpkgs_core and rules_nixpkgs_cc to ensure reproducible and incremental builds for external system packages.

Tokens
27.2K
Snippets
81
Records
109
Agent score
63%

What's inside rules_nixpkgs

  1. Overview of rules_nixpkgs

    master
    rules_nixpkgs allows you to use Nix and the Nixpkgs package set to import external dependencies (such as system packages) into Bazel hermetically. This ensures that if a dependency version changes, Bazel correctly rebuilds only the affected targets, maintaining fully reproducible and incremental builds.
  2. Rules for importing a Java toolchain from Nixpkgs

    master
    This project provides Bazel rules to import Java toolchains directly from a Nixpkgs repository. This allows Bazel to use the specific JDK versions and configurations managed by Nix, ensuring consistency between the Nix environment and the Bazel build process.
  3. Use Nix-provided Toolchains in Bazel

    master

    rules_nixpkgs allows you to use Nix-built packages as drop-in Bazel toolchains.

    How it works

    • Definition: Toolchains are defined similarly to Nix packages (using attribute paths, Nix files, or inline expressions).
    • Registration: The preferred pattern is for the language-specific module (e.g., rules_nixpkgs_cc) to automatically register the toolchain. This avoids requiring users to manually use use_repo, which can lead to unpredictable repository names.
    • Interface: The system strives for a consistent interface across different languages, allowing Nix toolchains to replace standard Bazel toolchains seamlessly.
  4. How nixpkgs_python_repository mimics rules_python

    master

    The nixpkgs_python_repository rule is intended to mimic the rules_python API. It provides a requirement function that creates labels in the format @{nixpkgs_python_repository_name}//:{package_name}.

    Warning: While depending on these labels directly works, the internal layout may change. For long-term stability, it is recommended to define and import your own requirement function to manage these labels.

  5. Use the local Bazel registry for rules_nixpkgs development

    master
    The rules_nixpkgs repository is split into multiple interdependent Bazel modules. Because standard Bazel overrides like local_path_override are restricted to the main module, and command-line --override_module flags do not support relative paths, a local Bazel registry is used for testing and local development. This allows you to manage dependencies between the various modules within the repository effectively.
  6. Understand the purpose of rules_nixpkgs

    master

    Bazel aims for hermetic and reproducible builds, but by default, it often relies on the host environment's global state (like the C++ compiler in PATH or system library versions). This can lead to non-reproducible builds where the same code produces different artifacts on different machines.

    rules_nixpkgs solves this by allowing Bazel to provision external dependencies and toolchains via the Nix package manager. This approach provides:

    • Fine-grained control: You specify exactly which packages and libraries are required.
    • Declarative dependencies: You specify what you need, and Nix handles the installation and versioning.
    • Minimized rebuilds: Adding a new dependency via rules_nixpkgs does not invalidate unrelated targets.
    • Elimination of containers: Nix provides the reproducibility benefits of a container/VM without the overhead of managing large Docker images or non-reproducible apt update commands.
  7. Identify module extensions by their import name

    master

    In Bzlmod, module extensions are identified by the Starlark module and name they are imported from, rather than by reference equality of the extension object.

    Warning: Do not attempt to re-export a module extension from a different location. If you import the same extension from two different .bzl files, Bazel will evaluate it multiple times in separate namespaces, which can lead to unexpected behavior or undetected cycles.

  8. Understand the rules_nixpkgs Bzlmod architecture

    master

    With the introduction of Bazel's Bzlmod dependency management, rules_nixpkgs has transitioned from using repository rules in WORKSPACE to using Module Extensions in MODULE.bazel.

    Module Separation

    To prevent dependency bloat and version conflicts, rules_nixpkgs is split into several specialized Bazel modules. Instead of one monolithic rule set, you should use the module relevant to your language to ensure you only pull in necessary transitive dependencies (e.g., using rules_nixpkgs_cc will pull in rules_cc, but won't pull in Go or Python rules).

    Core Components

    • Bazel Modules: Native Bazel projects (e.g., rules_nixpkgs_core, rules_nixpkgs_cc).
    • Module Extensions: Custom dependency types defined by modules that allow you to import Nix repositories, packages, and toolchains into your Bazel project via tags.
  9. Import Nix Packages via Module Extensions

    master

    Nix packages (Nix derivations or store paths) can be built, fetched, and imported into Bazel.

    Defining a Package

    When defining a package via a module extension tag, you can specify:

    • Nix Attribute Path: The path within a Nix repository (e.g., pkgs.hello). If not provided, it defaults to the tag name.
    • Inline Nix Expression: An expression that provides the attribute. Defaults to import <nixpkgs> { config = {}; overlays = []; } if no file is provided.
    • Local File: A local .nix file providing the attribute.
    • Nix Command-line Options: Optional configuration for the Nix build.

    Dependencies and Imports

    • Dependencies: Packages can depend on a Nix repository or a set of Nix repositories mapped to NIX_PATH entries.
    • BUILD Files: You can specify whether to use a default BUILD file or a custom BUILD file (provided as an inline string or a source file).
  10. Compile non-C++ languages using a C++ toolchain

    master

    You can use a C++ toolchain to compile non-C++ libraries (e.g., using Clang/LLVM for CUDA or HIP code). There are two ways to achieve this:

    1. Disable C++ mode in the toolchain: Pass cc_lang = "none" in the nixpkgs_cc_configure call. Then, in your cc_library or cc_binary rule, provide the appropriate compiler flags via copts (e.g., copts="-x cuda").
    2. Override the toolchain language: Use nixpkgs_cc_configure(..., cc_lang = "cuda") to set the toolchain's primary language mode.
    # Method 1: Disable C++ mode and use copts
    nixpkgs_cc_configure(..., cc_lang = "none")
    
    cc_library(
        name = "my_cuda_lib",
        copts = "-x cuda",
        ...
    )
    
    # Method 2: Set toolchain language directly
    nixpkgs_cc_configure(..., cc_lang = "cuda")
  11. Understand Bzlmod module extension constraints in rules_nixpkgs

    master

    The rules_nixpkgs implementation for Bzlmod is subject to several Bazel-specific constraints that affect how you manage Nix repositories and packages:

    • No Composition: Module extensions cannot invoke or read tags from other module extensions. For example, a toolchain tag cannot automatically discover a package tag to generate imports. You must handle re-use at the repository rule level or pass required repositories as labels.
    • Global Scope: Module extensions are evaluated globally. If two different Bazel modules request a Nix repository with the same tag name (e.g., "nixpkgs"), the extension must either unify them into a single workspace or generate unique names to avoid collisions.
    • Restricted Visibility: External workspaces generated by a module extension are only automatically visible to other workspaces generated by that same extension. To use a Nix repository or package in a different module, you must explicitly import it using a use_repo stanza.
    • No Direct Output: Module extensions cannot generate persistent, labeled Bazel outputs (like files) that are accessible to other rules. They can only generate files for internal use (e.g., for a dependency resolver). To expose information externally, the extension must invoke a repository rule that writes the data to a public location (like a Starlark constant).