aspect_rules_py

repository·main·Indexed 40 days ago

https://github.com/aspect-build/rules_py

A high-performance, hermetic Python ruleset for Bazel that replaces rules_python. It leverages uv for fast dependency resolution and python-build-standalone for robust interpreter provisioning.

Tokens
30.6K
Snippets
95
Records
120
Agent score
100%

What's inside rules_py

  1. Overview of uv-pyproject-cases

    main

    The uv-pyproject-cases directory serves as an umbrella end-to-end (e2e) testing suite for aspect_rules_py. It is designed to provide regression coverage for various pyproject.toml, setup.py, and setup.cfg configurations that the pep517_whl/build_helper.py must correctly route.

    Each case in this suite follows this pattern:

    1. Pins a real PyPI package via a central uv.lock.
    2. Forces the package to be resolved from its source distribution (sdist) using the [tool.uv].no-binary-package configuration.
    3. Executes a smoke py_test that imports the built package. The primary assertion is that the sdist build succeeds.
  2. Overview of aspect_rules_py

    main

    aspect_rules_py is a high-performance alternative to rules_python for Bazel. It provides drop-in replacements for py_binary, py_library, and py_test designed for modern Python workflows, large-scale monorepos, and Remote Build Execution (RBE).

    Key advantages include:

    • Native uv integration: Uses uv.lock directly for blazing-fast, sandboxed dependency resolution.
    • Hermeticity: Uses isolated Python execution (-I flag) and Bash-based launchers to prevent host environment leakage.
    • Idiomatic Layouts: Uses a standard site-packages symlink tree for seamless IDE compatibility (VSCode, PyCharm).
    • Cross-Platform: Supports native platform transitions for building multi-architecture OCI images and cross-compiling wheels.
    • Virtual Dependencies: Allows swapping implementations at the binary level or upgrading dependencies independently within a monorepo.
  3. Configure Python bootstrapping with bootstrap_template

    main

    The bootstrap_template is a File containing code responsible for the initial startup of a program. It handles locating the interpreter, setting environment variables, and invoking the entry point.

    The behavior of substitutions within this template depends on the --bootstrap_impl flag:

    --bootstrap_impl=system_python substitutions:

    • %is_zipfile%: 1 if prepended to a zipfile, 0 otherwise.
    • %python_binary%: Path to the target Python interpreter (absolute path, runfiles-relative path, or a name on PATH).
    • %target%: The name of the target.
    • %workspace_name%: The name of the workspace.
    • %coverage_tool%: Path to the coverage tool.
    • %import_all%: Boolean string.
    • %imports%: Colon-delimited runfiles-relative paths.
    • %main%: The entry point.
    • %shebang%: The shebang to use.

    --bootstrap_impl=script substitutions:

    • %is_zipfile%, %python_binary%, %target%, %workspace_name%, %shebang% (same as above).
    • %python_binary_actual%: The path to the interpreter that %python_binary% actually invokes (used for zip builds).
    • %stage2_bootstrap%: A runfiles-relative path to the stage 2 bootstrap.
  4. Use virtual dependencies in libraries

    main

    Virtual dependencies allow a py_library to declare a dependency on a package name without requiring a direct Bazel label. The actual Bazel target for that package is resolved later in the py_binary or py_test that consumes the library using the resolutions attribute.

    # In a library
    py_library(
        name = "greet_lib",
        srcs = ["greet.py"],
        virtual_deps = ["cowsay"],
    )
    
    # In a binary consuming the library
    py_binary(
        name = "app",
        srcs = ["main.py"],
        deps = [":greet_lib"],
        resolutions = {
            "cowsay": "@pypi//cowsay",
        },
    )
  5. Enable free-threaded Python runtime mode

    main

    Free-threaded mode (PEP 803) is available for Python 3.13 and newer. It is a separate axis from the standard build_config.

    How to enable:

    1. Globally: Use the build setting --@aspect_rules_py//py/private/interpreter:freethreaded=true during bazel build.
    2. Per-target: Set the freethreaded = True attribute on py_binary or other Python targets.

    Important constraints:

    • Free-threaded interpreters cannot load standard (GIL) native extensions. You must ensure all native dependencies use the t ABI (e.g., cp313t).
    • When inheriting a mode, the default is None, which follows the caller's mode.
    # Global enable
    bazel build --@aspect_rules_py//py/private/interpreter:freethreaded=true //...
    # Per-target enable
    py_binary(
        name = "free_threaded_app",
        srcs = ["app.py"],
        python_version = "3.13",
        freethreaded = True,
    )
  6. Understand the `aspect_rules_py//uv` mental model

    main

    The uv hub acts as a central repository that wraps multiple internal dependency group repositories (often called "spoke" repos).

    • Hub Repository: A central target (e.g., @pypi) that provides access to all requirements.
    • Requirement Targets: Accessing a library is done via the hub and the package name (e.g., @pypi//requests:requests).
    • Entrypoints: Accessing a specific entrypoint is done via the package name and the entrypoint name (e.g., @pypi//jinja2-cli/entrypoints:jinja2-cli).

    Important Constraints:

    • Hub requirement targets are incompatible with dependency group configurations in which that requirement is not defined.
    • If a package is gated by an environment marker (e.g., sys_platform == 'win32'), the hub uses a select() alias. On platforms where the marker is false, it selects an internal empty or empty_whl target, which contributes no sources or dependencies to ensure the build remains total across all platforms.
    @pypi                                     # Your UV built hub repository
    @pypi//requests:requests                  # The library for a requirement
    @pypi//jinja2-cli/entrypoints:jinja2-cli  # A requirement's declared entrypoint
  7. Manage PEP 735 dependency groups

    main

    Each [dependency-groups] entry in your pyproject.toml registers as a named group. You can switch between these groups at the hub level or the target level.

    Global default group

    Set a default group for the entire workspace in your .bazelrc:

    common --@pypi//dep_group=dummy

    Target-specific groups

    Use the dep_group attribute on py_binary or py_test to select a specific group:

    py_binary(
       name = "say_vendored",
       srcs = ["__main__.py"],
       deps = ["@pypi//cowsay"],
       dep_group = "vendored_say",
    )

    Group-specific dependency helpers

    To ensure a target uses all dependencies belonging to its active dep_group, use the group_deps() helper. This returns a select() that automatically resolves based on the target's dep_group.

    load("@pypi//:defs.bzl", "group_deps")
    
    py_binary(
        name = "all_vendored",
        dep_group = "vendored_say",
        deps = group_deps(),  # resolves to vendored_say's dependencies
    )

    For macros that need to inspect membership during the loading phase, use group_deps_for(name), which returns the membership of an explicit group as sorted Label values.

  8. Understand the PyRuntimeInfo structure

    main

    The PyRuntimeInfo provider contains information about a Python runtime, as returned by the py_runtime rule. It describes how a Python interpreter is accessed and how it should be bootstrapped to run targets within Bazel.

    A Python runtime can be one of two types:

    1. Platform runtime: Accesses a system-installed interpreter at a known absolute filesystem path (interpreter_path).
    2. In-build runtime: Points to a File that acts as the interpreter (interpreter). This can be a prebuilt binary, a checked-in interpreter, or one built from source.

    Warning: PyRuntimeInfo is an unstable public API. It may change frequently and has weaker compatibility guarantees.

    load("@aspect_rules_py//py:defs.bzl", "PyRuntimeInfo")
  9. Configure stage 2 bootstrapping with stage2_bootstrap_template

    main

    The stage2_bootstrap_template is a Python code template that runs under the desired interpreter. It orchestrates the call to the program's actual main code and is responsible for affecting the runtime state (like sys.path or enabling coverage) so that the program works correctly under Bazel.

    Available substitutions during expansion:

    • %main%: Runfiles-relative path to the program's actual main file (.py or .pyc).
    • %coverage_tool%: Runfiles-relative path to the coverage library's entry point (empty string if disabled).
    • %import_all%: String True if all repositories in runfiles should be added to sys.path, otherwise False.
    • %imports%: Colon-delimited string of runfiles-relative paths to add to sys.path.
    • %target%: The name of the target.
    • %workspace_name%: The name of the workspace.
  10. Configure zipapp support with zip_main_template

    main

    The zip_main_template is a Python code template used as the top-level __main__.py file for a zip file (per PEP 441). This allows the zip file to be executed directly via a Python interpreter.

    Note: py_binary-generated zip files are self-executing and skip calling __main__.py.

    Available substitutions during expansion:

    • %stage2_bootstrap%: A runfiles-relative string to the stage 2 bootstrap file.
    • %python_binary%: Path to the target Python interpreter (absolute, runfiles-relative, or PATH name).
    • %workspace_name%: The name of the workspace for the built target.
  11. Compatibility with rules_python

    main

    The aspect_rules_py interpreter provisioning is designed to coexist with rules_python using the following mechanisms:

    • Toolchain Registration: Uses the standard @bazel_tools//tools/python:toolchain_type, making these interpreters compatible with all existing Python rules.
    • Flag Synchronization:
      • The @rules_python//python/config_settings:python_version flag is kept in sync with Aspect's version flag via build transitions.
      • The @rules_python//python/config_settings:py_freethreaded flag is synchronized with Aspect's free-threading setting.
      • Best Practice: Set both flags to the same value in .bazelrc to prevent the configuration from moving into a second configuration subtree.
    • Runtime Compatibility: File-based runtimes registered with rules_python's py_runtime / py_runtime_pair are usable by rules_py rules. However, system interpreters (those using interpreter_path instead of an in-build interpreter file) are not supported; rules_py requires a registered, in-build interpreter.
    • Exec Tools: Build actions (like wheel installation) resolve @aspect_rules_py//py/private/toolchain:exec_tools_toolchain_type, which is registered by interpreters.toolchain().
    • Incremental Migration: You can migrate from rules_python to aspect_rules_py by replacing python.toolchain() calls with interpreters.toolchain() and removing the rules_python interpreter configuration.