Cargo-Rail

repository·main·Indexed 42 days ago

https://github.com/loadingalias/cargo-rail

A Rust workspace engine designed to optimize development workflows through verified compiler result reuse, precise affected-work planning, and automated release management. It provides tools for Rust source-surface analysis, dependency coherence, exact-SHA releases, and split/sync operations. Version 0.28.1 includes a local and remote caching system (CAS) to reuse verified compiler results across CI, IDEs, and local environments.

Tokens
38.9K
Snippets
104
Records
180
Agent score
53%

What's inside Cargo-Rail

  1. The mutation boundary and write sequence

    main

    All planned workspace edits (including those from unify and surface) must follow a strict sequence to ensure safety and recoverability:

    1. Capture: Capture the relevant source and repository state.
    2. Plan: Build a deterministic plan with exact authorized paths.
    3. Revalidate: Revalidate the captured assumptions immediately before writing.
    4. Apply: Apply only the planned changes and persist recovery evidence when needed.

    Note on boundaries: Cache writes, compiler acquisition journals, and installation profiles operate on their own validation and recovery boundaries; they do not pass through the standard workspace mutation plan.

  2. How compiler-backed Unify and Surface work

    main

    When performing unify or surface operations that require compiler support, Cargo-Rail follows a five-step acquisition engine:

    1. Schedule: Build a deterministic target and feature schedule from the captured workspace.
    2. Validate Cache: Admit only identity-matched, complete cached evidence.
    3. Execute: Run remaining views using bounded process slots, work permits, sandboxes, output, and artifact storage.
    4. Persist: Persist resumable progress before dispatch and cancel complete process trees after a failure.
    5. Integrate: Integrate completed work in a deterministic order, regardless of the order in which workers finished.
  3. Consume the Cargo-Rail machine contract (JSON plan)

    main

    The cargo rail plan --json command produces a schema-owned value (v9) that executors can consume.

    Key Fields in the JSON Plan:

    • identity: Content-derived decision identity.
    • inputs: Bindings for comparison, source, Cargo, configuration, toolchain, platform, catalog, and evidence.
    • work: A map of tagged required or skipped decisions.
    • required: A sorted projection of required work IDs.
    • work.NAME.scope: The selector attached to required work (e.g., work.NAME.scope.selection.cargo_args provides an argument array).

    Executor Workflow:

    1. Validate the complete plan and required projection.
    2. Select one known required work item.
    3. Lower only that item's typed scope.
    4. Run the reader's checkout verification in the execution workspace.
    5. Start the executor only after verification succeeds.
  4. Understand the Cargo-Rail Caching Layers

    main

    Cargo-Rail uses a multi-layered caching strategy to preserve work. Each layer provides a different level of authority and result type:

    • Cargo L0: Uses Cargo fingerprints and incremental state. Allows Cargo to skip fresh work.
    • Local L1: Stores exact compiler actions, results, and bytes. Restores a single compiler result locally.
    • Remote L2: Uses a machine-owned provider to share verified result packs. Results enter L1 before being restored.
    • Distributed miss: Uses a pinned worker capability and validated response to execute an eligible miss remotely.
    • Compiler evidence: Uses identity-matched, revalidated observations and typed facts to allow Unify or Surface to skip acquisitions.

    Note: A lookup is not authority. If cache evidence is missing, stale, ambiguous, unsupported, or corrupt, the system falls back to standard compiler execution with a stable miss or bypass reason.

  5. Understand Cargo-Rail benchmark scenarios

    main

    The local benchmark runner simulates four distinct build states to measure cache effectiveness. Each sample resets the target and result-cache state to the following configuration:

    ScenarioState at the timed build
    coldEmpty target and result cache; no seed build
    rebuildSeeded result cache; seed target removed
    freshnessSeeded result cache and target retained
    source-editSeeded state retained; source changed to produce 120 instead of 119
  6. How Cargo-Rail planning and execution work

    main

    Cargo-Rail provides an exact scope for executors (like Cargo, nextest, or CI) by combining source/config changes, target ownership, dependency edges, and observed inputs.

    The Workflow:

    1. Planning: cargo rail plan generates a plan containing exact cargo_args arrays for every required work decision.
    2. Execution: Instead of using path globs, pass the exact cargo_args provided by the planner to your executor.
    3. Validation: Cross-process consumers must validate the contract (v9) and verify that the current HEAD and captured source match the saved plan before executing selectors.

    Example Workflow for CI (GitHub Actions): Use the loadingalias/cargo-rail-action@v10 to run the planner and extract arguments for nextest.

    - uses: loadingalias/cargo-rail-action@v10
      id: rail
    
    - name: Test affected packages
      if: contains(fromJSON(steps.rail.outputs.required-work), 'cargo.test')
      shell: bash
      env:
        PLAN_FILE: ${{ steps.rail.outputs.plan-file }}
      run: |
        ARGS_FILE="$(mktemp "$RUNNER_TEMP/cargo-rail-args.XXXXXX")"
        cargo-rail-action plan cargo-args "$PLAN_FILE" cargo.test > "$ARGS_FILE" || exit "$"
        CARGO_ARGS=()
        while IFS= read -r -d '' argument; do CARGO_ARGS+=("$argument"); done < "$ARGS_FILE"
        rm -- "$ARGS_FILE"
        cargo nextest run "${CARGO_ARGS[@]}" --locked
    cargo-rail-action plan cargo-args "$PLAN_FILE" cargo.test
  7. Understand Cargo-Rail Policy Families

    main

    Cargo-Rail organizes its configuration into several distinct policy families that govern different aspects of the workspace lifecycle. Understanding these families is key to knowing which configuration section to use for a specific goal:

    • targets: Defines additional target triples used for target-aware workspace decisions.
    • [unify]: Manages dependency and manifest policies (e.g., MSRV, dependency resolution).
    • [surface]: Manages compiler-derived reachability and visibility policies.
    • [release]: Manages release intent, changelogs, and authority for external effects (like Git or registry publication).
    • [plan.work.NAME]: Defines positive inputs and selector shapes for repository-owned work.
    • [crates.NAME]: Defines per-crate policies for splitting, releasing, and changelogs.
  8. Understand the Cargo-Rail workflow model

    main

    Cargo-Rail operates using a model of separate, specialized workflows that share captured infrastructure but do not invoke one another implicitly. This separation ensures that each workflow owns a specific domain of responsibility and does not overstep its authority.

    WorkflowOwnsDoes not own
    planChanged inputs, Cargo impact, named-work decisions, typed selectorsTask execution
    unifyDependency diagnostics and manifest mutation plansGeneral Cargo execution
    surfaceCompiler-derived reachability and visibility plansSource lints outside its compiler contract
    cacheVerified compiler facts and resultsCargo freshness or incremental compilation
    change / releaseReviewed release intent and durable publication stateForge, registry, or Git implementation
    split / syncCrate ownership, history mapping, and conflict receiptsGeneral repository synchronization
  9. How compiler reuse works in Cargo-Rail

    main

    Cargo-Rail caches verified compiler results rather than raw target directories. Each hit revalidates the compiler action, inputs, environment, dependencies, outputs, and stored bytes before presenting the result to Cargo.

    Caching Layers:

    • Cargo L0: Cargo's own freshness and incremental compilation remain authoritative.
    • Local L1: Reuses verified results across Cargo, nextest, Just, IDEs, and CI.
    • Remote L2: Shares verified results via AWS S3, Cloudflare R2, or Azure Blob Storage.
    • Distributed miss: Automatically places new results once sufficient evidence of a material win is collected.

    Key Behaviors:

    • cargo clean does not delete the local Cargo-Rail CAS (Content Addressable Storage), allowing an empty target tree to still reuse verified work.
    • Local storage has a default 10 GiB limit. Cargo-Rail manages this by removing the oldest eligible action authorities when the limit is reached.
    • Use CARGO_RAIL_CACHE=off to run a cold build without interacting with the CAS.
  10. How Cargo-Rail manages workspace authority

    main

    Cargo-Rail uses a WorkspaceContext to maintain a single, authoritative view of the workspace. This context owns:

    • Source capture and effective configuration
    • Cargo metadata and the dependency graph
    • Lockfiles and toolchains
    • Targets and repository boundaries

    Key behaviors:

    • Snapshot-bound commands: Commands that rely on a snapshot will revalidate the relevant live state immediately before performing any writes.
    • Boundary revalidation: A command can only cross a named live boundary (such as Git readiness or registry state) if it revalidates that boundary at the exact moment the operation is performed.
  11. Quickstart: Enable compiler reuse, planning, and surface analysis

    main

    To get started with Cargo-Rail, follow these three primary workflows:

    1. Enable local compiler reuse

    Set up the local cache to allow Cargo to reuse verified compiler results.

    cargo rail cache setup --check
    cargo rail cache setup
    cargo rail cache ready
    cargo rail cache status

    2. Inspect affected work

    Generate a plan to see exactly which packages or targets are affected by changes in your current branch.

    cargo rail plan

    3. Audit Rust surface reachability

    Analyze the workspace to find dead public declarations or excessive visibility.

    # Prepare the analysis (may install rustc-dev)
    cargo rail surface --prepare
    
    # Check for issues with explanations
    cargo rail surface --check --explain
    
    # Attempt to fix visibility (dry-run first)
    cargo rail surface --fix --dry-run --explain
    cargo rail cache setup
    cargo rail plan
    cargo rail surface --prepare
  12. Debug incorrect work selection in a plan

    main

    If cargo rail plan selects the wrong work, use --explain for human-readable output or --json for programmatic inspection.

    When inspecting the JSON output via jq, verify these fields in order:

    1. inputs.base, inputs.head, and inputs.head_commit: Ensure the intended comparison base and head are correct.
    2. changes: Verify the expected file, Cargo, and configuration deltas.
    3. work.NAME.state: Check if the item is required or skipped. Required items include a .cause and refer to the top-level evidence map via .evidence entries.
    4. work.NAME.scope: For required items, check the package, target, or variant selector used by the executor.

    Note: If the plan is correct but execution is wrong, the issue lies with the consumer that lowered the typed scope. Always transfer the exact plan across CI jobs to avoid discrepancies caused by differing environments (toolchains, catalogs, etc.).

    cargo rail plan --explain
    cargo rail plan --json | jq '{inputs, changes, required, work, evidence}'